Skip to main content

v2.0.0 to v3.0.0

This guide provides details for the upgrade from v2.0.0 to v3.0.0.

caution

If you are NOT using PostgreSQL database, you should only follow the steps described in the Update branch to v3 section and skip the rest of the guide.

Update branch to v3​

Pull the changes from v3.0.0-stargate (v0.42.x/stargate) or v3.0.0 (v0.44.x and above) tag to your branch depending on the cosmos sdk version being used in your project. 

$ git fetch --tags
$ git checkout <your chain base>
$ git checkout -b merge/v3.0.0-stargate
$ git merge base-v3.0.0-stargate

Fix all merge conflicts and then install the latest BDJuno version 

$ make install

Migrate config file to v3​

Once installed you will need to migrate config.yaml file to v3. To do that run: 

$ bdjuno migrate v3

This will create additional sections inside the .yaml file for table partition and hasura actions config.
Don't panic if you see the message pops up: partition batch size is set to 0, skipping migration as this is expected. You will need to modify the values of the newly added partition_size and partition_batch sections before starting the migration. Make sure you set both values to 0 if you are not using the PostgreSQL database and want to skip the migration part. Read Database Config Reference for more information.

Remove Hasura Actions service​

From v3.0.0 Hasura Actions have been merged with BDJuno' to run on a single start command. This improves the BDJuno setup making it easier for users to follow. Find out more details how to enable and configure Hasura Actions

If you run Hasura Actions as a background service you should now stop it and remove it from the system. 

$ sudo systemctl stop hasura-actions.service
$ sudo rm /etc/systemd/system/hasura-actions.service

Stop BDJuno​

Now you need to stop BDJuno by running CTRL + C, or if you are running it as a system service run:

$ sudo systemctl stop bdjuno.service

Update BDJuno service (optional)​

If you are running BDJuno as a background service, you will need to update it before restarting. Edit bdjuno.service file and replace parse with start command at the ExecStart line. 

$ sudo nano /etc/systemd/system/bdjuno.service
[Unit]
Description=BDJuno parser
After=network-online.target

[Service]
User=$USER
ExecStart=$GOPATH/bin/bdjuno start
Restart=always
RestartSec=3
LimitNOFILE=4096

[Install]
WantedBy=multi-user.target

Then reload the configuration files

$ sudo systemctl daemon-reload

Start BDJuno​

Do NOT start BDJuno if you're using PostgreSQL database and you are going to perform table migration. Proceed to the next step instead.

If you are not using the PostgreSQL database you can now start BDJuno and continue the parsing as usual

$ bdjuno start

If you are running it as a background service run

$ sudo systemctl start bdjuno.service

Migration to PostgreSQL table partitioning​

caution

Skip the rest of this guide if you are NOT using the PostgreSQL database or if you don't want to perform the migration to PostgreSQL table partitioning.

In order to efficiently query the database with large amount of transactions and messages, we've implemented PostgreSQL Table Partitioning feature starting from v3.0.0 which turns the transaction and message tables into partitioned tables and creates partitions by a certain interval of heights set inside config.yaml file.

Prepare for the migration​

  1. Update partition values inside config.yaml file. 
database:
    # How many blocks for each partition. 
    # In this example partition is created per 100,000 blocks.
    partition_size: 100000
    
    # How many rows of transactions are migrated per batch 
    partition_batch: 1000

Read Database Config Reference for more information.

  1. If the chain implements custom messages that are different to standard cosmos messages, you will need to update the custom account parser (optional).

To update custom account parser follow the steps below:

  1. Fork Juno repository.
  2. Clone forked juno repository to your local machine.
$ git clone https://github.com/forbole/juno
$ cd juno
  1. Open juno repository in code editor of your choice and add your custom account parser inside juno/database/migrate/utils/utils.go
// Take desmos chain as example:
var CustomAccountParser = []string{
  "sender", "receiver", "user", "counterparty", "blocker", "blocked",
}
  1. Push the changes to github and copy the commit hash.
  2. Change directory back to bdjuno
$ cd ~/bdjuno
  1. Open go.mod file and replce the juno version with the commit hash you just copied.
  2. Run go mod tidy to reload the packages.
  3. Run make install to install the latest changes.
caution

Your custom Juno version should only be used during the migration process. When the migration is completed you should revert to using previous Juno version.

Start the migration​

Please ensure BDJuno is stopped before starting the migration.

We recommend setting up the migration process as a system service and being patient as the migration might take some time, depending on the database size.

$ sudo tee /etc/systemd/system/bdjuno-migrate-v3.service > /dev/null <<EOF
[Unit]
Description=BDJuno Table Partition Migration v3
After=network-online.target

[Service]
User=$USER
ExecStart=$GOPATH/bin/bdjuno migrate v3
LimitNOFILE=4096
Restart=no

[Install]
WantedBy=multi-user.target
EOF

Then you need to enable and start the service:

$ sudo systemctl enable bdjuno-migrate-v3.service
$ sudo systemctl start bdjuno-migrate-v3.service

BDJuno will now start migration which will:

  • rename transaction & message tables to transaction_old & message_old
  • rename related transaction and message indexes
  • create new transaction & message tables with partition
  • create indexes for new transaction and message table
  • migrate rows with data by batch from old to new tables
  • drop old function messages_by_address and create a new one

Start BDJuno​

While the migration is ongoing, you can already restart BDJuno to start parsing the latest data and it will be stored inside new partitioned tables:

$ bdjuno start

If you are running it as a background service run

$ sudo systemctl start bdjuno.service

Hasura​

Since v3.0.0, Hasura actions' handlers can use custom endpoints other than http://localhost:3000. Hasura will read from the ACTION_BASE_URL environmental variable as the action handler's base URL. To update it, you will need to re-create a new docker container, specifying the variable with the -e flag:

-e ACTION_BASE_URL="Base URL for Hasura Actions Handlers"

For more details, check out Installing Hasura

You will also need to re-apply hasura metadata to match the latest table setup:

$ cd /path/to/bdjuno/hasura
$ hasura metadata apply --endpoint <your-endpoint> --admin-secret <hasura_password>

Migration completed​

Once all transactions and messages have been migrated to new partitioned tables the migration is completed. For the safety reasons it does not drop old tables transaction_old and message_old. You might want to drop them after the data consistency is verified and they are no longer needed. 

DROP TABLE transaction_old;
DROP TABLE message_old;

Revert (optional)​

In any case that the migration fails, you can always revert the changes without losing any data. You might find the below SQL script helpful for altering tables and indexes back to the original names. Remember to drop the new transaction and message tables, along with the messages_by_address function.

Please ensure BDJuno is stopped prior to running the script. 
ALTER TABLE transaction_old RENAME TO transaction;
ALTER INDEX transaction_old_pkey RENAME TO transaction_pkey;
ALTER INDEX transaction_old_hash_index RENAME TO transaction_hash_index;
ALTER INDEX transaction_old_height_index RENAME TO transaction_height_index;
ALTER TABLE transaction RENAME CONSTRAINT transaction_old_height_fkey TO transaction_height_fkey;

ALTER TABLE message_old RENAME TO message;
ALTER INDEX message_old_involved_accounts_addresses RENAME TO message_involved_accounts_addresses;
ALTER INDEX message_old_transaction_hash_index RENAME TO message_transaction_hash_index;
ALTER INDEX message_old_type_index RENAME TO message_type_index;
ALTER TABLE message RENAME CONSTRAINT message_old_transaction_hash_fkey TO message_transaction_hash_fkey;

CREATE FUNCTION messages_by_address(
    addresses TEXT[],
    types TEXT[],
    "limit" BIGINT = 100,
    "offset" BIGINT = 0)
    RETURNS SETOF message AS
$$
SELECT message.transaction_hash, message.index, message.type, message.value, message.involved_accounts_addresses
FROM message
         JOIN transaction t on message.transaction_hash = t.hash
WHERE (cardinality(types) = 0 OR type = ANY (types))
  AND addresses && involved_accounts_addresses
ORDER BY height DESC
LIMIT "limit" OFFSET "offset"
$$ LANGUAGE sql STABLE;

When the tables are altered back, install and run your previous BDJuno branch which will continue to parse the data from the height before the migration started.

Last updated on