Celo Rosetta
A monitoring server for celo-blockchain
What is Celo Rosetta?
Celo Rosetta is an RPC server that exposes an API to:
- Query Celo's Blockchain
- Obtain Balance Changing Operations
- Construct Airgapped Transactions
With a special focus on getting balance change operations, Celo Rosetta provides an easy way to obtain changes that are not easily queryable using
the celo-blockchain rpc; such as:
- Gas Fee distribution
- Gold transfers (internal & external). Taking in account Tobin Tax
- Epoch Rewards Distribution
- LockedGold & Election Operations
RPC endpoints
Rosetta exposes the following endpoints:
POST /network/list
: Get List of Available Networks
POST /network/status
: Get Network Status
POST /network/options
: Get Network Options
POST /block
: Get a Block
POST /block/transaction
: Get a Block Transaction
POST /mempool
: Get All Mempool Transactions
POST /mempool/transaction
: Get a Mempool Transaction
POST /account/balance
: Get an Account Balance
POST /construction/metadata
: Get Transaction Construction Metadata
POST /construction/submit
: Submit a Signed Transaction
For an understanding of inputs & outputs check servicer.go
Command line arguments
Arguments are described in the help output of the CLI.
Every argument can be defined using environment variables using ROSETTA_
prefix; and replacing .
for _
; for example:
ROSETTA_DATADIR="/my/dir"
ROSETTA_GETH_NETWORK="alfajores"
Running the Rosetta RPC Server
Running the Rosetta RPC Server from scratch will take a very long time to sync and require at least 1.5TB of storage space, since it runs a full archive node in the background. While it may be possible to run the Construction API in the future with a non-archive node, this is still required by the Rosetta spec for the Data API implementation in order to perform balance reconciliation.
Version 1: Running from rosetta
source code
You will need the following three repositories cloned locally:
You also need the following dependencies to be met:
Running Rosetta
Prerequisites:
- Checkout the rosetta version that you want and run
make all
.
- Find the
celo-blockchain
version in the rosetta go.mod
file. Look for a line containing github.com/celo-org/celo-blockchain
the version comes after separated by a space.
- Checkout
celo-blockchain
at the version specified in rosetta's go.mod
and run make geth
- Replace
<NETWORK>
with one of alfajores
(developer testnet), baklava
(validator testnet) or mainnet
.
- Replace
<PATH-TO-DATADIR>
below, which is the location for the celo-blockchain data directory (the directory does not need to exist before passing it in).
The data directory is network specific so when switching networks you will also need to change the data directory.
Then run:
go run main.go run \
--geth.network <NETWORK> \
--geth.binary ../celo-blockchain/build/bin/geth \
--geth.syncmode full \
--geth.gcmode archive \
--datadir <PATH-TO-DATADIR>
You should start to see continuous output looking something like this:
INFO [01-28|14:09:03.434] Press CTRL-C to stop the process
--nousb --rpc --rpcaddr 127.0.0.1 --rpcport 8545 --rpcvhosts localhost --syncmode full --gcmode archive --rpcapi eth,net,web3,debug,admin,personal --ipcpath <YourPathToRosetta>/rosetta/envs/alfajores/celo/geth.ipc --light.serve 0 --light.maxpeers 0 --maxpeers 1100 --consoleformat term
INFO [01-28|14:09:05.110] Detected Chain Parameters chainId=44787 epochSize=17280
INFO [01-28|14:09:05.120] Starting httpServer listen_address=:8080
INFO [01-28|14:09:05.120] Resuming operation from last persisted block srv=celo-monitor block=0
INFO [01-28|14:09:05.121] SubscriptionFetchMode:Start srv=celo-monitor pipe=header_listener start=1
...
INFO [01-28|14:09:25.731] Stored 1000 blocks srv=celo-monitor pipe=persister block=1000 registryUpdates=0
You can stop the service and restart by re-running just the last command above (go run main.go
... )
Version 2: Running Rosetta Docker Image
Prerequisites:
- Install and run
docker
(tested with version 19.03.12
)
Rosetta is released as a docker image: us.gcr.io/celo-testnet/rosetta
. All versions can be found on the registry page. Within the docker image, we pack the rosetta
binary and also the geth
binary from celo-blockchain
. Rosetta will run both.
The command below runs the Celo Rosetta RPC server for alfajores
:
export RELEASE="latest" # or specify a release version
# folder for rosetta to use as data directory (saves rosetta.db & celo-blockchain datadir)
export DATADIR="${PWD}/datadir"
mkdir $DATADIR
docker pull us.gcr.io/celo-testnet/rosetta:$RELEASE
docker run --name rosetta --rm \
-v "${DATADIR}:/data" \
-p 8080:8080 \
us.gcr.io/celo-testnet/rosetta:$RELEASE \
run \
--geth.network alfajores \
--geth.syncmode full \
--geth.gcmode archive
To run this for a different network, change the geth.network
flag from alfajores
to mainnet
or baklava
.
Airgap Client Guide
The Celo Rosetta Airgap module is designed to facilitate signing transactions, parameterized by contemporaenous network metadata, in an offline context.
Examples of this metadata include:
- network wide state like "gas price minimum"
- argument specific state like vote amount "effect on validator priority queue"
AirGapServer {
ObtainMetadata(TxArgs): TxMetadata
SubmitTx(Tx): Status
}
AirGapClient {
ConstructTxFromMetadata(TxMetadata): Tx
SignTx(Tx, PrivateKey): Tx
}
Custody: Staking and Voting
For a documentation resource, please see the custody docs.
For a code resource, please see the examples.
Developer Guide
Setup
In addition to the dependencies listed above under the instructions for running from rosetta
source code, you also need:
openapi-generator
To re-generate rpc scaffold (install link)
Build Commands
Important commands:
make all
: Builds project (compiles all modules), same as go build ./...
make test
or go test ./...
to run unit tests
Interaction with Celo Core Contracts
Rosetta uses kliento to interact with the necessary Celo Core Contracts.
How to run rosetta-cli-checks
Note that running these checks is most likely infeasible for most people on
mainnet because you will need at least 1.5 terrabytes (as of 2022/11) of space
and several days to be able to sync the chain. Under the hood, the service is
syncing a full archive node, which takes a long time. The construction service
needs to reach the tip before submitting transactions. The data checks will
take a while to complete as well (likely a couple of days on a normal laptop
with the current settings) as they reconcile balances for the entire chain.
- Install the
rosetta-cli
according to the instructions. (Note that on Mac, installing the rosetta-cli
to /usr/local/bin
or adding its location to you $PATH
will allow you to call rosetta-cli
directly on the command line rather than needing to provide the path to the executable). Current testing has been done with v0.5.16
of the rosetta-cli
.
- Run the Rosetta service in the background for the respective network (currently only alfajores for both Data and Construction checks)
- Run the CLI checks for alfajores as follows:
# alfajores; specify construction or data
rosetta-cli check:construction --configuration-file PATH/TO/rosetta/rosetta-cli-conf/testnet/cli-config.json
How to generate bootstrap_balances.json
This is only necessary for running the data checks if it has not already been created for the particular network. Here's how to generate this for alfajores (for another network, specify the appropriate genesis block URL and output path):
go run examples/generate_balances/main.go \
https://storage.googleapis.com/genesis_blocks/alfajores \
rosetta-cli-conf/testnet/bootstrap_balances.json
Releasing rosetta
Versioning
Rosetta uses semantic versioning and contains 3 distinct
interfaces that should be taken into account when setting the version. Having
so many interfaces exported from this repo makes it likely that most changes
will constitute a breaking change in at least one interface.
- The rosetta HTTP API exposed by the services in this repo.
- The CLI exposed by the rosetta binary.
- All exported code in the repo, since this repo is imported as a module by Coinbase projects.
Breaking changes constitute any change that could cause a downstream consumer's
code to break. E.g:
- Any change that for a given API request results in different response bytes.
(This seems quite strict, but in the absence of any framework for classifying
breaking changes (that would need to be communicated to and agreed upon by
downstream consumers) we have to assume that any change in the bytes of a
response for a given request could break downstream consumers.)
- Any changes in CLI flags or default values, except if a new flag has been
added which has no effect unless explicitly set.
- Any changes to exported code in this repo or changes that would change the
value returned by code exported in this repo.
- Bear in mind that breaking changes in the API of celo-blockchain can imply
breaking changes for this repo, for example changing the value returned by gas
estimation, or adding a field to a response. Also note that celo-blockchain
does not follow semantic versioning, so when updating it you will need to
investigate all changes included in an update and understand how they affect
rosetta to see if any are breaking.
Changes that are not breaking and are not bug fixes are considered minor
versions, there will likely be very few of these.
Changes that are not breaking and are bug fixes are considered patch versions,
it seems quite likely bug fixes could also be breaking.
Making a release
- Ensure all required changes are merged to master.
- Create a PR to update the
MiddlewareVersion
in ./service/rpc/versions.go
set the version to the version of the release and merge to master.
- Update the master branch on your local repo and run
./scripts/set_tag_and_build_docker_images.sh
this will tag the current
commit with the MiddlewareVersion
and build and push docker images for this
version and print out the tags of the docker images.
- In github create a release for the aforementioned git tag, describe all
changes and breaking changes and add the tags for the docker images built by
the script.