{"id":20218209,"url":"https://github.com/numberfour/s3cmd","last_synced_at":"2026-04-20T06:08:41.662Z","repository":{"id":138345744,"uuid":"1614559","full_name":"NumberFour/s3cmd","owner":"NumberFour","description":null,"archived":false,"fork":false,"pushed_at":"2011-08-19T09:22:19.000Z","size":565,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":36,"default_branch":"master","last_synced_at":"2025-03-03T11:35:45.779Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://s3tools.org/s3cmd","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NumberFour.png","metadata":{"files":{"readme":"README","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2011-04-14T14:53:00.000Z","updated_at":"2017-06-05T17:44:45.000Z","dependencies_parsed_at":"2023-03-13T10:54:23.802Z","dependency_job_id":null,"html_url":"https://github.com/NumberFour/s3cmd","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/NumberFour/s3cmd","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NumberFour%2Fs3cmd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NumberFour%2Fs3cmd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NumberFour%2Fs3cmd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NumberFour%2Fs3cmd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NumberFour","download_url":"https://codeload.github.com/NumberFour/s3cmd/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NumberFour%2Fs3cmd/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32035278,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-20T00:18:06.643Z","status":"online","status_checked_at":"2026-04-20T02:00:06.527Z","response_time":94,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T06:37:36.974Z","updated_at":"2026-04-20T06:08:41.619Z","avatar_url":"https://github.com/NumberFour.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"S3cmd tool for Amazon Simple Storage Service (S3)\n=================================================\n\nAuthor:\n    Michal Ludvig \u003cmichal@logix.cz\u003e\n\nS3tools / S3cmd project homepage:\n    http://s3tools.org\n\nS3tools / S3cmd mailing lists:\n    * Announcements of new releases:\n        s3tools-announce@lists.sourceforge.net\n\n    * General questions and discussion about usage\n        s3tools-general@lists.sourceforge.net\n\n    * Bug reports\n        s3tools-bugs@lists.sourceforge.net\n\nAmazon S3 homepage:\n    http://aws.amazon.com/s3\n\n!!!\n!!! Please consult INSTALL file for installation instructions!\n!!!\n\nWhat is Amazon S3\n-----------------\nAmazon S3 provides a managed internet-accessible storage \nservice where anyone can store any amount of data and \nretrieve it later again. Maximum amount of data in one \n\"object\" is 5GB, maximum number of objects is not limited.\n\nS3 is a paid service operated by the well known Amazon.com\ninternet book shop. Before storing anything into S3 you\nmust sign up for an \"AWS\" account (where AWS = Amazon Web \nServices) to obtain a pair of identifiers: Access Key and\nSecret Key. You will need to give these keys to S3cmd. \nThink of them as if they were a username and password for\nyour S3 account.\n\nPricing explained\n-----------------\nAt the time of this writing the costs of using S3 are (in USD):\n\n$0.15 per GB per month of storage space used\n\nplus\n\n$0.10 per GB - all data uploaded\n\nplus\n\n$0.18 per GB - first 10 TB / month data downloaded\n$0.16 per GB - next 40 TB / month data downloaded\n$0.13 per GB - data downloaded / month over 50 TB\n\nplus\n\n$0.01 per 1,000 PUT or LIST requests\n$0.01 per 10,000 GET and all other requests\n\nIf for instance on 1st of January you upload 2GB of \nphotos in JPEG from your holiday in New Zealand, at the \nend of January you will be charged $0.30 for using 2GB of\nstorage space for a month, $0.20 for uploading 2GB\nof data, and a few cents for requests. \nThat comes to slightly over $0.50 for a complete backup \nof your precious holiday pictures.\n\nIn February you don't touch it. Your data are still on S3 \nservers so you pay $0.30 for those two gigabytes, but not\na single cent will be charged for any transfer. That comes \nto $0.30 as an ongoing cost of your backup. Not too bad.\n\nIn March you allow anonymous read access to some of your\npictures and your friends download, say, 500MB of them. \nAs the files are owned by you, you are responsible for the \ncosts incurred. That means at the end of March you'll be \ncharged $0.30 for storage plus $0.09 for the download traffic \ngenerated by your friends.\n\nThere is no minimum monthly contract or a setup fee. What \nyou use is what you pay for. At the beginning my bill used\nto be like US$0.03 or even nil.\n\nThat's the pricing model of Amazon S3 in a nutshell. Check\nAmazon S3 homepage at http://aws.amazon.com/s3 for more \ndetails.\n\nNeedless to say that all these money are charged by Amazon \nitself, there is obviously no payment for using S3cmd :-)\n\nAmazon S3 basics\n----------------\nFiles stored in S3 are called \"objects\" and their names are\nofficially called \"keys\". Since this is sometimes confusing\nfor the users we often refer to the objects as \"files\" or\n\"remote files\". Each object belongs to exactly one \"bucket\".\n\nTo describe objects in S3 storage we invented a URI-like\nschema in the following form:\n\n    s3://BUCKET\nor\n    s3://BUCKET/OBJECT\n\nBuckets\n-------\nBuckets are sort of like directories or folders with some \nrestrictions:\n1) each user can only have 100 buckets at the most, \n2) bucket names must be unique amongst all users of S3, \n3) buckets can not be nested into a deeper hierarchy and \n4) a name of a bucket can only consist of basic alphanumeric \n   characters plus dot (.) and dash (-). No spaces, no accented\n   or UTF-8 letters, etc. \n\nIt is a good idea to use DNS-compatible bucket names. That\nfor instance means you should not use upper case characters.\nWhile DNS compliance is not strictly required some features\ndescribed below are not available for DNS-incompatible named\nbuckets. One more step further is using a fully qualified\ndomain name (FQDN) for a bucket - that has even more benefits.\n\n* For example \"s3://--My-Bucket--\" is not DNS compatible.\n* On the other hand \"s3://my-bucket\" is DNS compatible but \n  is not FQDN.\n* Finally \"s3://my-bucket.s3tools.org\" is DNS compatible \n  and FQDN provided you own the s3tools.org domain and can\n  create the domain record for \"my-bucket.s3tools.org\".\n\nLook for \"Virtual Hosts\" later in this text for more details \nregarding FQDN named buckets.\n\nObjects (files stored in Amazon S3)\n-----------------------------------\nUnlike for buckets there are almost no restrictions on object \nnames. These can be any UTF-8 strings of up to 1024 bytes long. \nInterestingly enough the object name can contain forward\nslash character (/) thus a \"my/funny/picture.jpg\" is a valid\nobject name. Note that there are not directories nor\nbuckets called \"my\" and \"funny\" - it is really a single object \nname called \"my/funny/picture.jpg\" and S3 does not care at \nall that it _looks_ like a directory structure.\n\nThe full URI of such an image could be, for example:\n\n    s3://my-bucket/my/funny/picture.jpg\n\nPublic vs Private files\n-----------------------\nThe files stored in S3 can be either Private or Public. The \nPrivate ones are readable only by the user who uploaded them\nwhile the Public ones can be read by anyone. Additionally the\nPublic files can be accessed using HTTP protocol, not only\nusing s3cmd or a similar tool.\n\nThe ACL (Access Control List) of a file can be set at the \ntime of upload using --acl-public or --acl-private options \nwith 's3cmd put' or 's3cmd sync' commands (see below).\n\nAlternatively the ACL can be altered for existing remote files\nwith 's3cmd setacl --acl-public' (or --acl-private) command.\n\nSimple s3cmd HowTo\n------------------\n1) Register for Amazon AWS / S3\n   Go to http://aws.amazon.com/s3, click the \"Sign up\n   for web service\" button in the right column and work \n   through the registration. You will have to supply \n   your Credit Card details in order to allow Amazon \n   charge you for S3 usage. \n   At the end you should have your Access and Secret Keys\n\n2) Run \"s3cmd --configure\"\n   You will be asked for the two keys - copy and paste \n   them from your confirmation email or from your Amazon \n   account page. Be careful when copying them! They are \n   case sensitive and must be entered accurately or you'll \n   keep getting errors about invalid signatures or similar.\n\n3) Run \"s3cmd ls\" to list all your buckets.\n   As you just started using S3 there are no buckets owned by \n   you as of now. So the output will be empty.\n\n4) Make a bucket with \"s3cmd mb s3://my-new-bucket-name\"\n   As mentioned above the bucket names must be unique amongst \n   _all_ users of S3. That means the simple names like \"test\" \n   or \"asdf\" are already taken and you must make up something \n   more original. To demonstrate as many features as possible\n   let's create a FQDN-named bucket s3://public.s3tools.org:\n\n   ~$ s3cmd mb s3://public.s3tools.org\n   Bucket 's3://public.s3tools.org' created\n\n5) List your buckets again with \"s3cmd ls\"\n   Now you should see your freshly created bucket\n\n   ~$ s3cmd ls\n   2009-01-28 12:34  s3://public.s3tools.org\n\n6) List the contents of the bucket\n\n   ~$ s3cmd ls s3://public.s3tools.org\n   ~$ \n\n   It's empty, indeed.\n\n7) Upload a single file into the bucket:\n\n   ~$ s3cmd put some-file.xml s3://public.s3tools.org/somefile.xml\n   some-file.xml -\u003e s3://public.s3tools.org/somefile.xml  [1 of 1]\n    123456 of 123456   100% in    2s    51.75 kB/s  done\n\n   Upload a two directory tree into the bucket's virtual 'directory':\n\n   ~$ s3cmd put --recursive dir1 dir2 s3://public.s3tools.org/somewhere/\n   File 'dir1/file1-1.txt' stored as 's3://public.s3tools.org/somewhere/dir1/file1-1.txt' [1 of 5]\n   File 'dir1/file1-2.txt' stored as 's3://public.s3tools.org/somewhere/dir1/file1-2.txt' [2 of 5]\n   File 'dir1/file1-3.log' stored as 's3://public.s3tools.org/somewhere/dir1/file1-3.log' [3 of 5]\n   File 'dir2/file2-1.bin' stored as 's3://public.s3tools.org/somewhere/dir2/file2-1.bin' [4 of 5]\n   File 'dir2/file2-2.txt' stored as 's3://public.s3tools.org/somewhere/dir2/file2-2.txt' [5 of 5]\n\n   As you can see we didn't have to create the /somewhere\n   'directory'. In fact it's only a filename prefix, not \n   a real directory and it doesn't have to be created in\n   any way beforehand.\n\n8) Now list the bucket contents again:\n\n   ~$ s3cmd ls s3://public.s3tools.org\n                          DIR   s3://public.s3tools.org/somewhere/\n   2009-02-10 05:10    123456   s3://public.s3tools.org/somefile.xml\n\n   Use --recursive (or -r) to list all the remote files:\n\n   ~$ s3cmd ls s3://public.s3tools.org\n   2009-02-10 05:10    123456   s3://public.s3tools.org/somefile.xml\n   2009-02-10 05:13        18   s3://public.s3tools.org/somewhere/dir1/file1-1.txt\n   2009-02-10 05:13         8   s3://public.s3tools.org/somewhere/dir1/file1-2.txt\n   2009-02-10 05:13        16   s3://public.s3tools.org/somewhere/dir1/file1-3.log\n   2009-02-10 05:13        11   s3://public.s3tools.org/somewhere/dir2/file2-1.bin\n   2009-02-10 05:13         8   s3://public.s3tools.org/somewhere/dir2/file2-2.txt\n\n9) Retrieve one of the files back and verify that it hasn't been \n   corrupted:\n\n   ~$ s3cmd get s3://public.s3tools.org/somefile.xml some-file-2.xml\n   s3://public.s3tools.org/somefile.xml -\u003e some-file-2.xml  [1 of 1]\n    123456 of 123456   100% in    3s    35.75 kB/s  done\n\n   ~$ md5sum some-file.xml some-file-2.xml\n   39bcb6992e461b269b95b3bda303addf  some-file.xml\n   39bcb6992e461b269b95b3bda303addf  some-file-2.xml\n\n   Checksums of the original file matches the one of the \n   retrieved one. Looks like it worked :-)\n\n   To retrieve a whole 'directory tree' from S3 use recursive get:\n\n   ~$ s3cmd get --recursive s3://public.s3tools.org/somewhere\n   File s3://public.s3tools.org/somewhere/dir1/file1-1.txt saved as './somewhere/dir1/file1-1.txt'\n   File s3://public.s3tools.org/somewhere/dir1/file1-2.txt saved as './somewhere/dir1/file1-2.txt'\n   File s3://public.s3tools.org/somewhere/dir1/file1-3.log saved as './somewhere/dir1/file1-3.log'\n   File s3://public.s3tools.org/somewhere/dir2/file2-1.bin saved as './somewhere/dir2/file2-1.bin'\n   File s3://public.s3tools.org/somewhere/dir2/file2-2.txt saved as './somewhere/dir2/file2-2.txt'\n\n   Since the destination directory wasn't specified s3cmd \n   saved the directory structure in a current working \n   directory ('.'). \n\n   There is an important difference between:\n      get s3://public.s3tools.org/somewhere\n   and\n      get s3://public.s3tools.org/somewhere/\n   (note the trailing slash)\n   S3cmd always uses the last path part, ie the word\n   after the last slash, for naming files.\n \n   In the case of s3://.../somewhere the last path part \n   is 'somewhere' and therefore the recursive get names\n   the local files as somewhere/dir1, somewhere/dir2, etc.\n\n   On the other hand in s3://.../somewhere/ the last path\n   part is empty and s3cmd will only create 'dir1' and 'dir2' \n   without the 'somewhere/' prefix:\n\n   ~$ s3cmd get --recursive s3://public.s3tools.org/somewhere /tmp\n   File s3://public.s3tools.org/somewhere/dir1/file1-1.txt saved as '/tmp/dir1/file1-1.txt'\n   File s3://public.s3tools.org/somewhere/dir1/file1-2.txt saved as '/tmp/dir1/file1-2.txt'\n   File s3://public.s3tools.org/somewhere/dir1/file1-3.log saved as '/tmp/dir1/file1-3.log'\n   File s3://public.s3tools.org/somewhere/dir2/file2-1.bin saved as '/tmp/dir2/file2-1.bin'\n\n   See? It's /tmp/dir1 and not /tmp/somewhere/dir1 as it \n   was in the previous example.\n\n10) Clean up - delete the remote files and remove the bucket:\n\n   Remove everything under s3://public.s3tools.org/somewhere/\n\n   ~$ s3cmd del --recursive s3://public.s3tools.org/somewhere/\n   File s3://public.s3tools.org/somewhere/dir1/file1-1.txt deleted\n   File s3://public.s3tools.org/somewhere/dir1/file1-2.txt deleted\n   ...\n\n   Now try to remove the bucket:\n\n   ~$ s3cmd rb s3://public.s3tools.org\n   ERROR: S3 error: 409 (BucketNotEmpty): The bucket you tried to delete is not empty\n\n   Ouch, we forgot about s3://public.s3tools.org/somefile.xml\n   We can force the bucket removal anyway:\n\n   ~$ s3cmd rb --force s3://public.s3tools.org/\n   WARNING: Bucket is not empty. Removing all the objects from it first. This may take some time...\n   File s3://public.s3tools.org/somefile.xml deleted\n   Bucket 's3://public.s3tools.org/' removed\n\nHints\n-----\nThe basic usage is as simple as described in the previous \nsection.\n\nYou can increase the level of verbosity with -v option and \nif you're really keen to know what the program does under \nits bonet run it with -d to see all 'debugging' output.\n\nAfter configuring it with --configure all available options\nare spitted into your ~/.s3cfg file. It's a text file ready\nto be modified in your favourite text editor.\n\nFor more information refer to:\n* S3cmd / S3tools homepage at http://s3tools.org\n* Amazon S3 homepage at http://aws.amazon.com/s3\n\nEnjoy!\n\nMichal Ludvig\n* michal@logix.cz\n* http://www.logix.cz/michal\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnumberfour%2Fs3cmd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnumberfour%2Fs3cmd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnumberfour%2Fs3cmd/lists"}