All add-ons currently require PHP 7.4 or greater.
On July 4th 2024 PHP 8.2 will be the new minimum requirement for all add-ons. Expect any add-on released after that date to require 8.2 or greater. Some releases may not immediately take advantage of 8.x specific features in PHP, which means you might, be able to continue using new releases in PHP 7.4, however, if you experience an error the first thing you should do is update to PHP 8.2 then create a support ticket if the error persists.
Publisher does not support the Fluid field type. Please do not contact asking when support will be available.
If you purchased an add-on from expressionengine.com, be sure to visit boldminded.com/claim to add the license to your account here on boldminded.com.
Ticket: Import existing files in sub-directories of main upload location
| Status | Resolved |
| Add-on / Version | DataGrab 5.1.1 |
| Severity | |
| EE Version | 7.3.14 |
James Brown
Dec 20, 2023I’m sure this is me doing something wrong. I’m having issues importing into File field. The files themselves are being imported from a Wordpress site I’m migrating over, so it has that Wordpress structure of putting the files by directories for year/month. Ex: /wp-content/uploads/2023/12/...
I have uploaded that full /wp-content directory to the new server and made it an EE file location with the base location being /wp-content/uploads/ (see attached for setup). I want to have my import associate the existing file to the entry being created. In the xml import file I have the field value as the full url. Ex: https://phpstack-717800-4003432.cloudwaysapps.com/wp-content/uploads/2023/12/Thumb-752x500-1.png
I have tried a number of different combos on the settings but it isn’t quite doing what I’d hope. I tried specifing an upload folder and set Fetch files from URLs to Yes. But that ends up copying the files in question to the upload folder location into a single directory. I wouldn’t care that they get copied into 1 directory EXCEPT that some of the files end up having the same file name but different contents. Since Wordpress puts them in directories by year/month of upload they have unique URLs and no naming issues. But once they’re put into a single directory DataGrab thinks when it sees a filename existing with the same name that it’s already there and just uses that.
I tried specifying the upload folder and setting the Fetch files from URLs to No hoping that it would find that existing file in the upload folder’s subdirectory and associate it, but when I run the import, that just results in the File field being blank.
What is the proper method to import and link to an existing file that is in a sub directory of an upload folder?
BoldMinded (Brian)
That’s not a DG issue, that’s EE.
This is likely an EE issue too. Do you have the new File Manager enabled with sub-directories? My suspicion is that this is still an EE issue… I don’t think DG should have to be responsible for reading and creating multiple sub-directories before uploading a file. DG is just using EE’s own ee()->filemanager->save_file() and ee()->upload->raw_upload() methods.
The easiest thing for you might be to re-name the files, e.g. /wp-content/uploads/2023/12/Thumb-752x500-1.png to /wp-content/uploads/2023-12-Thumb-752x500-1.png and upload everything to the same directory.
James Brown
Hey Brian. I’m probably doing a poor job of explaining myself.
That first and second part you were mentioning wasn’t me saying was a DG issue, but just a reason why I couldn’t use that approach. I wasn’t looking for DG to create subdirectories.
I’m trying to figure out if it is possible to be able to link to existing files in an existing upload location that contains subdirectories. It may not be possible, but just trying to figure out my options. On the DG side when I’m configuring the File field there is the “Upload folder” is it maybe that DG only looks there for an existing file?
It would be difficult to rename all the files and update the export file since there’s hundreds of files. So that’s why I’m trying to see if there’s a way to make this work.
BoldMinded (Brian)
I haven’t done a lot of investigating on how EE handles the new subdirectories, so I’m not really sure I have an answer for you.
When you tried fetching the images from a remote URL into the upload folder, did you have the “Allow subfolders?” option on the upload directory enabled? It seems like EE should be able to manage this… it should see the slashes in the file name and create the necessary folder structure.
James Brown
Yea, it had the allow subfolders checked. My guess of what was happening is that the remote image was fetched, and then put into the Upload folder selected in DG. In most cases that would be totally fine, but this site has a number of their files named the same thing, just put in different subdirectories.
BoldMinded (Brian)
James are you able to share a sample of your import file? I just need 4 or 5 items to test the import with.
BoldMinded (Brian)
Comment has been marked private.
James Brown
Hey Brian- Sorry, was a bit away for the holidays. I can try and get you a demo file if that would help. Do you still need one?
I did end up resolving it in a bit of a hacky way. I wrote a perl script to recursively go through the directories and move the files to a single directory and append the subdirectories to the beginning of the file name. So /2022/12/file.png becomes 202212file.png. Then a little find and replace in my import file and I was good to go.
James
BoldMinded (Brian)
Try this build https://www.dropbox.com/scl/fi/h97uea6ay7dj09056sgpc/datagrab-5.1.1-create-file-sub-dirs-5a5deb07.zip?rlkey=rkwqmpg0ehl5f3zd6j0sz8j0c&dl=0
There is a new option for the File fields to tell it to create the sub-directories.
BoldMinded (Brian)
Closing this out b/c this is in the latest DataGrab release.