I’m a lazy blogger and it’s been a long pause, but here I go again…
I’ve been with Linux for quite some time. In fact, I’ve banished Windows from all my machines and I’m running only Linux now. So, I considered myself to be fairly familiar with the command line., however, surprises are still possible.
Today I was uploading large amount of files to a web server. Because of the way server is set, the application directories are writable only by sudo users and I didn’t find a way of making Filezilla use the sudo command. SCP wasn’t an option either. So I had to upload everything to my home directory on that machine and then login via SSH and move the files. Easy, right, it’s just:
sudo mv ~/files/* /var/www/app/files/
Well, not quite. This command returned an error:
bash: /bin/mv: Argument list too long
I checked what I’ve typed, mv expects two arguments, input file/folder and output file/folder. How could this error be reported then, I haven’t made an error.
Short online search gave me a hint, mv command can fail when it has to move too many files. I checked the folder where I’ve uploaded them, and there were several thousands. By the way, you can count the files in a directory with this command: “ls -l | wc -l”, subtract two from the number you get and that’s how many files you have in the current directory (wc -l counts lines, it will also count “.” and “..”).
So, what to do now, how to move files when mv command has failed and there is nothing else available. The dump way of solving this would be to feed mv one small subset of files, like this:
sudo mv ~/files/0* /var/www/app/files/
sudo mv ~/files/1* /var/www/app/files/
sudo mv ~/files/2* /var/www/app/files/
In my case, I had files with hex character set (0-9, a-f), so there were 16 possible combinations of the first character of the file name. There weren’t that many files in my case and this approach would have been possible, but it’s still tedious. In my opinion, the best way is to use rsync. Another good way is to use oneliner shell script, this approach is also useful for other actions, such as changing permissions.
Here’s an example of oneliner shell script:
for file in source/*; do cp $file dest/ ; done