Agnus Dei (jackal) wrote,
Agnus Dei
jackal

How to copy buckets in AWS S3

Copying from the old bucket "webapp-apache-logs" to the new bucket "webapp-logs" only the folders that start with pod01-webapp-06 (a single servers logs):

# s3funnel webapp-apache-logs list --list-prefix=pod01-webapp-06 -v -a MYKEY -s MYSECRET | s3funnel webapp-logs copy --source-bucket webapp-apache-logs -a MYKEY -s MYSECRET -t 30 -v
INFO     Using input from stdin
INFO     Using input from stdin
INFO     Starting pool with 30 threads.
INFO     Listing keys from marker:
INFO     Done listing bucket: webapp-apache-logs
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-30.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-30.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-08-07.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-08-07.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-22.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-22.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-19.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-19.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-08-06.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-08-06.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-26.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-26.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-25.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-25.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-08-09.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-08-09.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-27.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-27.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-08-01.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-08-01.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-08-05.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-08-05.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-08-02.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-08-02.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-08-04.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-08-04.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-08-10.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-08-10.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-18.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-18.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-08-03.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-08-03.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-20.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-20.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-08-08.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-08-08.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-08-11.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-08-11.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-31.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-31.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-28.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-28.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-29.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-29.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-21.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-21.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-21.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-21.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-20.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-20.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-24.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-24.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-access_log.2013-07-23.txt.gz to pod01-webapp-06/bradsdomain.com-access_log.2013-07-23.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-19.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-19.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-21.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-21.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-20.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-20.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-24.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-24.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-23.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-23.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-22.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-22.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-11.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-11.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-04.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-04.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-27.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-27.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-28.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-28.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-25.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-25.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-26.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-26.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-30.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-30.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-03.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-03.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-10.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-10.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-01.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-01.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-27.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-27.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-29.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-29.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-31.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-07-31.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-05.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-05.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-07.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-07.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-02.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-02.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-10.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-10.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-06.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-06.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-28.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-28.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-19.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-19.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-03.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-03.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-11.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-11.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-09.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-09.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-07-19.txt.gz to pod01-webapp-06/ipad_access_log.2013-07-19.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-07-22.txt.gz to pod01-webapp-06/ipad_access_log.2013-07-22.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-07-21.txt.gz to pod01-webapp-06/ipad_access_log.2013-07-21.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-07-20.txt.gz to pod01-webapp-06/ipad_access_log.2013-07-20.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-07-28.txt.gz to pod01-webapp-06/ipad_access_log.2013-07-28.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-07-23.txt.gz to pod01-webapp-06/ipad_access_log.2013-07-23.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-07-27.txt.gz to pod01-webapp-06/ipad_access_log.2013-07-27.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-08.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-08.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-07-24.txt.gz to pod01-webapp-06/ipad_access_log.2013-07-24.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-08-02.txt.gz to pod01-webapp-06/ipad_access_log.2013-08-02.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-07-25.txt.gz to pod01-webapp-06/ipad_access_log.2013-07-25.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-08-03.txt.gz to pod01-webapp-06/ipad_access_log.2013-08-03.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-07-29.txt.gz to pod01-webapp-06/ipad_access_log.2013-07-29.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-08-01.txt.gz to pod01-webapp-06/ipad_access_log.2013-08-01.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-04.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-error_log.2013-08-04.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-07-26.txt.gz to pod01-webapp-06/ipad_access_log.2013-07-26.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-08-04.txt.gz to pod01-webapp-06/ipad_access_log.2013-08-04.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-08-06.txt.gz to pod01-webapp-06/ipad_access_log.2013-08-06.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-08-05.txt.gz to pod01-webapp-06/ipad_access_log.2013-08-05.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-07-31.txt.gz to pod01-webapp-06/ipad_access_log.2013-07-31.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-07-30.txt.gz to pod01-webapp-06/ipad_access_log.2013-07-30.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-08-07.txt.gz to pod01-webapp-06/ipad_access_log.2013-08-07.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-08-10.txt.gz to pod01-webapp-06/ipad_access_log.2013-08-10.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-08-11.txt.gz to pod01-webapp-06/ipad_access_log.2013-08-11.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-08-08.txt.gz to pod01-webapp-06/ipad_access_log.2013-08-08.txt.gz
INFO     Copied: pod01-webapp-06/ipad_access_log.2013-08-09.txt.gz to pod01-webapp-06/ipad_access_log.2013-08-09.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-02.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-02.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-22.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-22.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-25.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-25.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-26.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-26.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-01.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-01.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-07.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-07.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-24.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-24.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-09.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-09.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-31.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-31.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-23.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-23.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-29.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-29.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-05.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-05.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-30.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-30.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-06.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-06.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-18.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-07-18.txt.gz
INFO     Copied: pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-08.txt.gz to pod01-webapp-06/bradsdomain.com-ssl-access_log.2013-08-08.txt.gz




Subscribe
  • Post a new comment

    Error

    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

  • 0 comments