crates-mirror

Crates Mirror

Download all crates on Rust official crates site and keep sync with it. This can be used to setup a static mirror site of https://crates.io. This can be used with cargo-mirror to make dependency download faster when building Rust project.

This fork adds more implementations of the crates-mirror

Requirements

• Good bandwidth (at least can access aws-s3 service of us region)
• Large hard disk (at least 30G, the current size is 18G, 2018-11-15.)

Go

How

1. Clone this repo: git clone https://github.com/EmilSodergren/crates-mirror.git

2. go get github.com/mattn/go-sqlite3

3. go install

4. Add a config.json file in your current directory, or place it wherever and pass the path as an argument.

5. Run the program $GOBIN/crates-mirror [/path/to/config.json]

   The config.json has the following structure:

   {
       "indexurl":"https://github.com/rust-lang/crates.io-index",
       "registrypath":"/path/to/the/crates.io-index",
       "cratespath":"/path/to/the/downloaded/crate/files",
       "dbpath":"/path/to/the/metadata/database",
       "updateindex":true    <--- if not present, will use true as default
   }

Python (The original)

• Python3 (yes python3, python2 is dead)

How

1. Clone this repo: git clone https://github.com/tennix/crates-mirror
2. Fire a python virtualenv:

   cd crates-mirror
   pyvenv env
   source env/bin/activate
   pip install -r requirements.txt
   

3. Run this program: python app.py
4. (Optional)Serve a mirror site:

   cd crates
   python -m http.server 8000
   

Note: for production, you should make this program auto-restarted when dies (supervisord like tools is needed). And also use a production web server (nginx, apache etc.) to serve the mirror site