admin action: loop through .csv file rows but when done unlink this file

[froot]

so long story short, I'm currently working on an AdminAction by which I can attach a .csv file, loop through the rows and do some more code on what I get from there. My problem is, I can never upload the same file again, i.e. cannot unlink the file it appears, which is quite annoying, particularly when developing (but also when actually using the AdminAction in the future), because it says it's already in the system and belongs to another field. I have to rename the file each time I did some changes and want to execute the action.

Where is that file actually? In the session variable? in a cache folder somewhere?

Anyways, my code starts somewhat like this

$url = config()->paths->assets.'cache/AdminActions/';
        
if(count($options['csv_upload'])) {
     $file = $options['csv_upload']->first();
     $fp = fopen($file->filename, 'r');
            
     while(($row = fgetcsv($fp)) !== false) {
          ...

and at the very end of the code, after all the different if-conditions and while-loops I do include…

fclose($fp);
unlink($file->filename);

but that doesn't work.

– hope you don't mind tagging you here @adrian

[adrian]

@fruid - can you please post the entire code for your action - should be an easy fix, but will be simpler if I have everything to test things.

[froot]

    protected function defineOptions() {

        return array(
            array(
                'name' => 'csv_upload',
                'label' => 'CSV upload',
                'description' => 'Upload a CSV file.',
                'type' => 'file',
                'extensions' => 'csv txt',
                'overwrite' => true,
                'maxFiles' => 1,
                'required' => true,
            ),
            array(
                'name' => 'template',
                'label' => 'Template',
                'description' => 'Choose the template you want to import content to',
                'type' => 'select',
                'required' => true,
                'options' => array(
                    '1' => 'article',
                    '2' => 'blog_repeater'
                ),
            ),
            array(
                'name' => 'deleteimages',
                'label' => 'delete current images',
                'description' => 'delete current images',
                'type' => 'checkbox'
            ),
            array(
                'name' => 'addimages',
                'label' => 'add new images',
                'description' => 'add new images',
                'type' => 'checkbox'
            )
        );

    }

    protected function executeAction($options) {
                
        $parent = '';
        $parent_title = '';
        $url = config()->paths->assets.'cache/AdminActions/';
        
        if(count($options['csv_upload'])) {
            $n = 0;
            $file = $options['csv_upload']->first();
            $fp = fopen($file->filename, 'r');
            
            if ($options['template'] != 2) : // different actions for different templates

                while(($row = fgetcsv($fp)) !== false) {
                    if(++$n === 1) : // skip the csv-file's first line
                        continue; 
                    endif;

                    if ($options['template'] == 1) :

                        list($parent_title, $title, $author, $summary, $body, $page_number, $img_folder) = $row;
                        $parent = pages()->get("template=magazine, title=$parent_title");
                        $name = sanitizer()->pageName($title, true);

                        if ($parent->id != 0) : // check if parent exists
                            if($parent->hasChildren("name=$name") == false) { // create the page if it doesn't exist
                                $p = new Page();
                                $p->template = 'article';
                                $p->parent = $parent;
                                $p->name = $name;
                                $this->message("Creating new page: $p->path");
                            } else { // ...or merely update it
                                $p = $parent->child("name=$name"); 
                                $this->message("Updating existing page: $p->path");
                            }
                        else:
                            $this->message("$parent_title has the wrong template");
                        endif;

                        // set the field's values according to the csv-input
						$p->title = $title;
                        $p->author = $author;
                        $p->summary = $summary;
                        $p->body = $body;
                        $p->page_number = $page_number;

                        $img_folder = sanitizer()->removeWhitespace($img_folder);
                        $img_field = $p->images;

                        if(!$parent->id) {
                            $this->message("row $n: Sorry parent $parent_title does not exist");
                            continue;
                        }

                        $specs = array( // prepare the import of the images via ftp
                            'sourceFolder' => $url.$img_folder.'/',
                            'destinationPage' => "$p->id",
                            'deleteFolder' => 0,
                            'field' => fields()->get('images')->id,
                        );

						// optionally DELETE all images before importing (i.e. replacing the images)
                        if ($options['deleteimages'] === 1) :
                            $img_field->deleteAll();
                            $this->message("images deleted");
                        endif;

						// optionally ADD new images
                        if ($options['addimages'] === 1) :
                            if ($options['template'] === 1 || $options['template'] === 2) :
                                modules()->get("ProcessAdminActions")->FtpFilesToPage($specs);
                                $this->message("images added");                
                            endif;
                        endif;

                        $p->of(false);
                        $p->save();                
                        $p->of(true);        
                    
                    endif;

                }

            else :
            
                while(($row = fgetcsv($fp)) !== false) {
                    if(++$n === 1) : // skip the csv-file's first line
                        continue; 
                    endif;

                    list($parent, $title, $date, $image, $caption, $img_folder) = $row;
                    $parent = pages()->get("name=blog");
                    $name = sanitizer()->pageName($title, true);

                    if($parent->hasChildren("name=$name") == false) { // create the page if it doesn't exist
                        $p = new Page();
                        $p->of(false);
                        $p->template = 'blog_repeater';
                        $p->parent = $parent;
                        $p->name = $name;
                        $this->message("Creating new page: $p->path");
                        $p->title = $title;                        
                        $p->date = $date;
                        $p->save();    
                        $this->successMessage = 'post created';
                    } else { // ...or merely update it
                        $this->message("Updating existing page: $p->path");
                        $p = $parent->child("name=$name"); 
                        $p->of(false);
                        $p->title = $title;
                        $p->date = $date;
                        $p->save();    
                        $this->successMessage = 'post updated';
                    }
                    
                    $p = $parent->child("name=$name"); 

                    if(!$parent->id) {
                        $this->message("row $n: Sorry parent $parent_title does not exist");
                        continue;
                    }
                                        
					// optionally DELETE all images before importing (i.e. replacing the images)
                    if ($options['deleteimages'] === 1) :
                        $p->of(false);
                        $p->image_and_caption->deleteAll();
                        $this->message("images deleted");
                        $p->save(image_and_caption);
                        $p->of(true);
                    endif;

					// optionally ADD new images
                    if ($options['addimages'] === 1) :
                        $p->of(false);
                        $img_folder = sanitizer()->removeWhitespace($img_folder);
                        $sourceFolder = $url.$img_folder.'/';
                        $item = $sourceFolder.$image;
                        $newItem = $p->image_and_caption->getNew();  
                        $newItem->save();
                        $newItem->post_image->add($item);
                        $newItem->ckeditor_caption = $caption;
                        $newItem->save();
                        $p->save(image_and_caption);
                        $p->of(true);
                        $this->successMessage = 'image added';
                    endif;                    
                }
            endif;                        

            return true;
            fclose($fp);

        } else {

            $this->failureMessage = 'Please upload a CSV file';
            return false;

        }
                
        fclose($fp);
		// the following are 3 of many attempts to solve the issue, none of them work :(
        $options['csv_upload']->unlink($file->getPathname());
        $options['csv_upload']->unlink($file->filename, $limitPath=true);
        $this->wire('files')->unlink($file->filename); 

    }

}

It's quite long (and it's even a bit shortened here in the post, because there's more elseif statement for very similar actions for different templates) and possibly redundant at this point but I'd rather get it all to work and then think about how to shorten it.

As you can see I make use of another AdminAction within this AdminAction, namely FtpFilesToPage. I don't see an issue with that though, works fine.

So either of these elseif subaction above face the same problem as described in the initial post of this thread.

I'm actually a step further, I realised all the 65 (!) csv files that I used to test my AdminAction with are stored in the AdminAction's /site/assets/files/1443/ folder (1443 only in my case obviously). So when I delete those, everything works as it should, just don't know how to have the script delete it automatically. ->unlink() is not helpful. Maybe $pages->get('1443')->deleteAll(); or something? Best practice?

Thanks for looking into that @adrian

[adrian]

@fruid - this works:

$this->wire('files')->unlink($file->filename);

but the problem is that in your example, your code never gets here because you have "return true" or "return false" in the if/else above this point.

I simplified things right down so obviously this doesn't handle the processing of the CSV file, but this is basically what you want so that you are unlinking before returning true.

    protected function executeAction($options) {

        $parent = '';
        $parent_title = '';
        $url = config()->paths->assets.'cache/AdminActions/';

        if(count($options['csv_upload'])) {
            $n = 0;
            $file = $options['csv_upload']->first();
            $fp = fopen($file->filename, 'r');
            fclose($fp);

            $this->wire('files')->unlink($file->filename);

            return true;

        } else {

            $this->failureMessage = 'Please upload a CSV file';
            return false;

        }

    }

 

[froot]

I tried exactly that just now, doesn't work though, unfortunately the .csv file stays in /site/assets/files/1443/ folder

[adrian]

1 minute ago, fruid said:

I tried exactly that just now, doesn't work though, unfortunately the .csv file stays in /site/assets/files/1443/ folder

Sorry, I'm not sure then - it's working here. I think perhaps you need to do some debugging at your end to make sure the unlink call is actually being made and also clarify that $file->filename contains the full path to the csv file. Also, try: $this->wire('files')->unlink('/fullpath/site/assets/files/1443/file.csv'); in the Tracy console just to test this in isolation. If that doesn't work, try a regular php unlink and if that doesn't work, check to make sure there aren't any file permission issues.

[froot]

@adrian you were exactly right, unlinking before returning was the issue. My bad, the reason it didn't work was a stupid mistake from my side not even worth mentioning.

AdminActions is awesome, now that I get the hang of it I can make those actions that I code and that I change the arguments of manually in the code easily usable for the client, no need for them to ever look at code, just clicky-clicky.

That said, I do still have a small issue that may or may not have to do with the module (probably not but I mention it anyway). When I do $img_field->deleteAll(); ($img_field being $p->images; see above) it deletes the images alright but leaves "orphaned images" (I think that's what it's called, i.e. images with 0KB in the page. Any ideas what that's about? I also tried ->removeVariations(); before ->deleteAll(); but no luck.

Thanks a lot!

[adrian]

1 hour ago, fruid said:

That said, I do still have a small issue that may or may not have to do with the module (probably not but I mention it anyway). When I do $img_field->deleteAll(); ($img_field being $p->images; see above) it deletes the images alright but leaves "orphaned images" (I think that's what it's called, i.e. images with 0KB in the page. Any ideas what that's about? I also tried ->removeVariations(); before ->deleteAll(); but no luck.

Looks like that is happening because you are turning off outputformatting after doing the deleteAll(). Do it beforehand and things should be fine.

[froot]

yes, again, the issue was something like that. Many thanks!

[froot]

sorry to reawaken this thread but it seems like I'm having the exact same issue again, with the same code in the same project.

This is my code after processing the CSV-file

$this->successMessage = "$count CSV rows were processed.";
fclose($fp);
$this->wire('files')->unlink($file->filename);
return true;

yet the file remains in the site/assets/files folder and the error is the same "file already in the system" 

Any more suggestions?

[adrian]

Did you check the file permissions and ownership? 

What happens if you make the unlink call directly (maybe from Tracy Console) - do you get a permissions error?

 

[froot]

Not sure how that works, haven't used the console yet.

I put 

$this->wire('files')->unlink($file->filename)

in the tracy console when looking at that specific admin action dialogue.
Then I hit "run" and it runs. But the file is not deleted and I get no errors or anything. 

[froot]

I gave the folder site/assets/files/4444/ where the CSV-files are stored 777 permissions, if that's what you meant. Doesn't help though.

 

[froot]

it actually works fine after all, must have been a permissions issue indeed OR I just had to refresh the modules OR the fact that some CSV-files with the same name where in the folder already (why they were there and not deleted remains a mistery). When I deleted these files, it worked fine and now no more issues so far.

I just wonder if this is right

fclose($fp);
$this->wire('files')->unlink($file->filename);

or should it be

$this->wire('files')->unlink($file->filename);
fclose($fp);

or if it even makes a difference.

[adrian]

I don't believe the order will matter, but having the fclose before the unlink makes more sense to me.