Solution: bash: /usr/local/bin/rsync: Argument list too long

This issue came from Unix limitation of command argument memory maximum size. And the solution is straightforward: don’t use <directory>/*, use <directory> instead. Here I show you how the issue came out and how I make it work again.

Issue

oracle@stbyserver ~]$ rsync -n -u -r -vz oracle@prodserver:/archive/prod/* /archive/stby/
bash: /usr/bin/rsync: Argument list too long
rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
rsync error: remote command could not be run (code 126) at io.c(463) [receiver=2.6.8]
Well, it hit the same wall as command ls does:
[oracle@prodserver ~]$ ls /archive/prod/*
-bash: /bin/ls: Argument list too long
 

Solution

[oracle@stbyserver ~]$ rsync -n -u -r -vz oracle@prodserver:/archive/prod /archive/
receiving file list ... done
prod/1_878494_893489722.dbf
...
sent 48 bytes  received 95579 bytes  27322.00 bytes/sec
total size is 10151739904  speedup is 106159.77
It is time to rsync again:
[oracle@stbyserver ~]$ rsync -avz oracle@prodserver:/archive/prod /archive/

Reason

Well, this limitation came from Unix. 

[oracle@prodserver ~]$ getconf ARG_MAX
131072  =>128Kbytes
[oracle@prodserver ~]$ cat /usr/include/linux/binfmts.h
#ifndef _LINUX_BINFMTS_H
#define _LINUX_BINFMTS_H

#include <linux/ptrace.h>
#include <linux/capability.h>

/*
 * MAX_ARG_PAGES defines the number of pages allocated for arguments
 * and envelope for the new program. 32 should suffice, this gives
 * a maximum env+arg of 128kB w/4KB pages!
 */
#define MAX_ARG_PAGES 32 

/* sizeof(linux_binprm->buf) */
#define BINPRM_BUF_SIZE 128

#endif /* _LINUX_BINFMTS_H */

About bigpang

Database Architect, Oracle Certified Master, Perl advocate and Python user
This entry was posted in Linux and tagged , , , , , . Bookmark the permalink.

4 Responses to Solution: bash: /usr/local/bin/rsync: Argument list too long

  1. Pingback: Solution: bash: /usr/local/bin/rsync: Argument list too long | 老庞闲话

  2. wasp digital says:

    Thanks! just saved me a couple hours of headbashing!

  3. barry41 says:

    You can also use Long Path Tool for such problems, it works good.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s