Running an Aspire Node
This document describes how one can set up their own Aspire “Federated Node” system, on Linux, Windows or OS X.
Explore: AspireAssets
Running an Aspire Node
Setting up a AspireWallet Node
Introduction
This document describes how one can set up their own Aspire “Federated Node” system, on Linux, Windows or OS X.
A Federated Node is a self-contained system that runs the some or all of the Aspire software stack, via Docker. Each system operates as a gAsp and Aspire “full node”. Using this toolset, one can generally get started running the Aspire software much quicker and more easily than a manual installation of the various components.
The document is primarily intended for power users and developers.
Running an Aspire Node
Setting up a AspireWallet Node
Node Services
Services run on a Federated Node include some or all of the following:
- aspire-server:
aspire-lib
+aspire-cli
. Implements support for the core Aspire protocol, via a provided REST API and command line interface. - aspireblock: Provides additional services (required by
aspirewallet
and potentially other services) beyond those offered in the API provided byaspire-server
. It features a full-fledged JSON RPC-based API, and has an extensible architecture to support custom plugins. - aspirewallet: The reference Web wallet for Aspire. This is a collection of HTML, CSS and javascript resources, served by
nginx
. - gaspd: Reference gAsp implementation, used by
aspire-server
to sync to the gAsp blockchain. - indexd-server: gAsp address index service. Maintains an updated database of UTXOs for usage in the Aspire services.
- nginx: Reverse proxies
aspirewallet
access. Not used withaspire-server
-only oraspireblock
-only nodes. - mongodb and redis: Used by
aspirerblock
.
Please note that Federated Node should not be installed on a system which already has one or more of conflicting services running on the ports used by Federated Node. The Federated Node install script checks that required ports are unused and exits to avoid conflict. If you have a non-essential Web, mongodb or other service running on the target system you can disable them or bind them to a different port to be able to pass the built-in check and avoid application conflicts.
Running an Aspire Node
Setting up a AspireWallet Node
Hardware / OS requirements
- Memory: 4GB RAM (
gaspd
,aspirewallet-server
only), 8GB+ RAM (full stack) - Disk space: The exact disk space required will be dependent on what services are run on the node:
- For
gasp
databases: ~100GB (mainnet), ~10GB (testnet) - For
aspire
andaspireblock
databases: ~4.2GB each
- For
- OS: Please note that Ubuntu Linux is the recommended OS at this time, as most of our testing is performed on it. Windows and OS X support is considered in BETA.
- Linux: We recommend Ubuntu 16.04 64-bit, but other, modern versions of Linux should work, as long as they support the newest released version of Docker
- Windows: Windows 7 or higher, or Server 2008 or higher. 64-bit required
- OS X: 10.8 “Mountain Lion” or higher
Running an Aspire Node
Pre-installation
Windows
NOTE: Installation on Windows is still in BETA state, and we cannot promise a fully-working environment. Please report any bugs you find.
- Python 3.5.x: Download and install the latest Python 3.5.x release. Make sure you check the box “Add Python 3.5 to PATH” on the first page. (If you get an error during installation, make sure your windows system is fully updated via Windows Update.)
- Docker: If using Windows 10, we recommend to install Docker for Windows. For all other versions of Windows, install Docker Toolbox.
- Git: Make sure
git
is installed. If not, install it from here (note that if using Docker Toolbox, it will install it by default).
If using Docker for Windows:
- After installing Docker for Windows, launch the “Docker” application and allow it to set itself up (a reboot may be required).
- Next, you will need to enable access to your host hard drive so that some of the shared volumes work properly. To do this, right click on the Docker Whale icon in your system tray. Then go to “Docker Settings” and then “Shared Drives”. Turn on access to the drive on which the
federatednode
folder will reside (most likely your “C” drive). - Finally, launch a command prompt as Administrator
- After installation completes, launch the “Docker Quickstart Terminal” and let it configure itself.
- Once this finishes, you will need to resize the Virtual Machine that Docker Toolbox uses to run the Docker containers. Note that it currently limits this VM to 1GB of memory and 20GB hard disk space total by default (shared across all containers). You will need to update this to at least 2 or 4GB memory and 50-100GB space. To do this, execute commands like the following (replacing the numbers in the second command as appropriate, based on the system requirements):
docker-machine rm default docker-machine create --driver virtualbox --virtualbox-disk-size "100000" --virtualbox-memory "4096" default
Then, relaunch the Docker Quickstart Terminal, and verify that docker ps
functions normally (if not, restart the system and try the command again).
Running an Aspire Node
Pre-installation
OS X
- Python 3.5.x: Download and install the latest Python 3.5.x release. Make sure you check the box “Add Python 3.5 to PATH” on the first page.
- Docker: If using OS X Yosemite or higher, we recommend to install Docker for Mac. For Older Macs, install Docker Toolbox.
- Git: Make sure
git
is installed. If not, install it from here (note that if using Docker Toolbox, it will install it by default).
If using Docker for Mac, launch the “Docker” application and allow it to set itself up, then open a terminal prompt.
If using Docker Toolbox, launch the “Docker Quickstart Terminal” once installation finishes, and follow the instructions on resizing your VM above.
Running an Aspire Node
Pre-installation
Linux
(Instructions are provided for Ubuntu Linux. Other Linuxes will be similar. Use a sudo-er account, but not root)
Update system & install dependencies
Make sure you have Python 3.5. (Ubuntu 14.04 for instance uses Python 3.4 by default), but 16.04 uses 3.5. If you have an Ubuntu version older than 3.4, you can update your Python with these instructions.
sudo apt-get update && sudo apt-get upgrade sudo apt-get -y install git curl coreutils
Install docker-ce and docker-compose (see here for more info, here we use v1.16.1):
sudo -i # become root curl -fsSL https://get.docker.com/ | sh curl -L https://github.com/docker/compose/releases/download/1.16.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose chmod +x /usr/local/bin/docker-compose exit # leave root shell
Running an Aspire Node
Installation
On Linux and OS X, install as a non-root sudo-er from home directory.
Clone and check out the code
On all OS, clone federatednode repo and enter cloned directory:
git clone https://github.com/AspireOrg/federatednode.git cd federatednode
On Linux and OS X:
sudo ln -sf `pwd`/fednode.py /usr/local/bin/fednode
On Windows (if using Docker Quickstart Terminal, a.k.a MINGW64):
mkdir ~/bin echo "python.exe \"`pwd`\\fednode.py\" \$*" > ~/bin/fednode chmod +x ~/bin/fednode
On Windows (if using Windows Command prompt):
> C:\Windows\fednode.bat echo python.exe %CD%\fednode.py \%*
Build and link the containers
Run the following command:
fednode install <CONFIG> <BRANCH>
Where <CONFIG>
is one of the following:
base
if you want to runaspire-server
andbitcoind
onlycounterblock
if you want to run everything inbase
, with the addition ofaspireblock
and its dependencies (mongodb
andredis
)full
if you would like to run a full federated node configuration, which is all services on the list above
And where <BRANCH>
is one of the following:
master
(stable and recommended)develop
(cutting edge, likely with bugs)
For example:
# install a base configuration for the master branch fednode install base master # install a full configuration for the develop branch fednode install full develop
In some cases (slow host, limited bandwidth), you may experience a failure to install due to download timeouts which happen because of network unstability. In that case consider changing Docker’s max-concurrent-downloads
value to 1 or 2 from default 3. To do that create a custom /etc/docker/daemon.json
daemon options file and restart Docker service.
As mentioned earlier, the install script may stop if ports used by Federated Node services are used by other applications. While it is not recommended to run Federated Node alongside production services, small changes can make the evaluation of Federated Node easier. For example you may change ports used by existing applications (or disable said applications) or run Federated Node inside of a virtual machine.
For example, the original mongodb can be reconfigured to listen on port 28018 and aspireblock’s mongodb can use the default port 27017. The Federated Node install script makes it possible to specify the interface used by its mongodb container (example below), but it currently does not have the ability to do this for other services or get around port conflicts.
fednode install --mongodb-interface 127.0.0.2 aspireblock master
Wait for initial sync
After installation, the services will be automatically started. To check the status, issue:
fednode ps
If you have existing instances of gAsp Core (either mainnet or testnet), at this point you could stop all services listed in fednode ps
output, change configuration files (of aspire and aspireblock, for example) and point them to your existing gAsp Core. Configuration files can be found in various service directories located under federatednode/config.
Once the containers are installed and running, keep in mind that it will take some time for gaspd
to download the blockchain data. Once this is done, aspire-server
will fully start and sync, followed by aspireblock
(if in use). At that point, the server will be usable.
You may check the sync status by tailing the appropriate service logs, e.g. for gAsp Core and Aspire server on mainnet:
fednode tail gasp fednode tail aspire
Once running, the system listens on the following ports:
aspire-server
: 4000/tcp (mainnet), 14000/tcp (testnet)aspireblock
: 4001/tcp (mainnet), 14001/tcp (testnet)
For aspire-server
, use RPC username rpc
and default password rpc
.
If aspirewallet
is installed, access to the following URLs will be possible:
http://<host>/
— directs tohttps
https://<host>/
– main production URL (uses minified JS/CSS)https://<host>/src/
– development URL (uses un-minified JS/CSS)
Running an Aspire Node
Post-installation Tasks
Ensure that your firewall software is enabled. If you want to provide access from external systems, you can allow through some or all of the appropriate ports. In addition, if you are running a node in a production scenario, it is recommended that you properly secure it.
You may also want to tighten ownership and permissions on all conf files in federatednode/config subdirectories, but keep in mind that you should be the only user with access to the operating system that runs Federated Node containers: Federated Node is not designed for shared OS environments.
Ubuntu Linux
Ubuntu Linux users can optionally run a little script that will issue a number of commands to assist with securing their systems:
cd extras/host_security sudo ./run.py
Note that this script will make several modifications to your host system as it runs. Please review what it does here before using it.
If you expect to run a busy Federated Node that requires aspireblock, you can consider making the following performance tweaks for mongodb and redis. Please do not make these changes to the host if you’re not comfortable with them because they impact not only Docker but the entire OS.
- Disable huge memory pages (for redis and mongodb): on Ubuntu 16.04 add
echo "never" > /sys/kernel/mm/transparent_hugepage/enabled
to /etc/rc.local and runsudo systemctl enable rc-local.service
. Reboot and check withcat /sys/kernel/mm/transparent_hugepage/enabled
(expected setting:[never]
). - Edit /etc/sysctl.conf (for redis): add
net.core.somaxconn = 511
andvm.overcommit_memory = 1
and runsudo sysctl -p
.
Running an Aspire Node
Administration
Checking status
To check the status of the containers, run:
fednode ps
Running an Aspire Node
Administration
Modifying configurations
Configuration files for the gasp
, aspire
and aspireblock
services are stored under federatednode/config/
and may be freely edited. The various locations are as follows:
gasp
: Seefederatednode/config/gasp/gasp.conf
gasp-testnet
: Seefederatednode/config/gasp/gasp.testnet.conf
indexd
: Seefederatednode/config/indexd/indexd.env.default
indexd-testnet
: Seefederatednode/config/indexd/indexd.testnet.env.default
aspire
: Seefederatednode/config/aspire/server.conf
aspire-testnet
: Seefederatednode/config/aspire/server.testnet.conf
aspireblock
: Seefederatednode/config/aspireblock/server.conf
aspireblock-testnet
: Seefederatednode/config/aspireblock/server.testnet.conf
redis
: shared service used for both mainnet and testnetmongodb
: shared service used for both mainnet and testnet
Remember: once done editing a configuration file, you must restart
the corresponding service. Also, please don’t change port or usernames/passwords if the configuration files unless you know what you are doing (as the services are coded to work together smoothly with specific values).
For example, a user with base setup (gAsp Core & Aspire Server) could make Aspire use existing gAsp Core by changing configuration files found under federatednode/config/aspire/ (backend-connect
in Aspire server configuration files and wallet-connect
in client configuration files.) At this point gAsp Core (mainnet and/or testnet) container(s) could be stopped and aspire server container restarted. If your existing gAsp Server allows RPC connections, with proper settings and correct RPC credentials in their configuration files, aspire (server), aspireblock and aspirewallet can all use it so that you don’t have to run gasp or gasp-testnet container.
Running an Aspire Node
Administration
Viewing/working with stored data
The various services use Docker named volumes to store data that is meant to be persistent:
gasp
andgasp-testnet
: Stores blockchain data in thefederatednode_gasp-data
volumeindexd
andindexd-testnet
: Stores index data in thefederatednode_indexd-data
volumeaspire
andaspire-testnet
: Stores Aspire databases in thefederatednode_aspire-data
volumeaspireblock
andaspireblock-testnet
: Stores Aspireblock asset info (images), etc in thefederatednode_aspireblock-data
volumemongodb
: Stores the databases foraspireblock
andaspireblock-testnet
in thefederatednode_mongodb-data
volume
Use docker volume inspect <volume-name>
to display volume location. See docker volume --help
for help on how to interact with Docker volumes.
Running an Aspire Node
Administration
Viewing logs
To tail the logs, use the following command:
fednode tail <service>
Or, to view the entire log, run:
fednode logs <service>
Where <service>
may be one the following, or blank to tail all services:
aspire
(aspire-server
mainnet)aspireblock
(aspireblock
mainnet)gasp
(gasp
mainnet)indexd
(indexd
mainnet)aspire-testnet
aspireblock-testnet
gasp-testnet
indexd-testnet
aspirewallet
Running an Aspire Node
Administration
Stopping and restarting containers
fednode stop <service> fednode start <service> fednode restart <service>
Where <service>
is one of the service names listed above, or blank for all services.
Note that redis and mongodb are shared services and need to run if either (mainnet or testnet) aspireblock container is running and shut down only if both aspireblock containers are not running.
Running an Aspire Node
Administration
Issuing a Single Shell Command
fednode exec <service> <CMD>
Where <service>
is one of the service names listed above, and <CMD>
is an arbitrary shell command.
For example:
fednode exec aspire aspire-client send --source=G2u4Vymr3bGTywjMQDgBkwAnazwQuDqzJG --destination=GAanCo9CJSomhUEy2YrhfXrU1PboBhFaBq --quantity=1.5 --asset=ASP fednode exec gasp-testnet bitcoin-cli getpeerinfo fednode exec aspireblock ls /root
Running an Aspire Node
Administration
Getting a shell in a container
fednode shell <service>
Where <service>
is one of the service names listed above.
Running an Aspire Node
Updating, rebuilding, uninstalling
Running an Aspire Node
Updating, rebuilding, uninstalling
Reparsing blockchain data
Both aspire-server
and aspireblock
read in blockchain data and construct their own internal databases. To reset these databases and trigger a reparse of this blockchain data for one of the services, run:
fednode reparse <service>
Where service is aspire
, aspire-testnet
, aspireblock
, or aspireblock-testnet
.
Running an Aspire Node
Updating, rebuilding, uninstalling
Rebuilding a service container
As a more extensive option, if you want to remove, rebuild and reinstall a container (downloading the newest container image/Dockerfile
and utilizing that):
fednode rebuild <service>
Where <service>
is one of the service names listed earlier, or blank for all services. Note that you are just looking to update the source code and restart the service, update
is a better option.
Running an Aspire Node
Updating, rebuilding, uninstalling
Uninstalling
To uninstall the entire fednode setup, run:
fednode uninstall
Running an Aspire Node
Component development
Other Developer Notes
- To run the
aspire-lib
test suite, execute:
fednode exec aspire "cd /aspire-lib/aspirelib; py.test --verbose --skiptestbook=all --cov-config=../.coveragerc --cov-report=term-missing --cov=./"
- If you are working on
aspirewallet
, you should browse the system using the/src/
subdirectory (e.g.https://myaspirewallet.bla/src/
). This avoids using precompiled sources. Once you are happy with your changes and ready to make them available to everyone that hits the server, runfednode update aspirewallet
, which will pull the newest repo code and repackage the web assets so that the code updates are then active fromhttps://myaspirewallet.bla/
. - Note that when you install the federated node system, HTTPS repository URLs are used by default for all of the repositories checked out under
src
byfednode.py
. To use SSH URIs instead, specify the--use-ssh-uris
to thefednode install
command.
Running an Aspire Node
AspireWallet-Specific
If you are setting up a AspireWallet server, you will next need to create a
aspirewallet.conf.json
configuration file. Instructions for doing that are detailed in the AspireWallet Configuration File section later in this document. Once creating this file, open up a web browser, and go to the IP address/hostname of the server. You will then be presented to accept your self-signed SSL certificate, and after doing that, should see the AspireWallet login screen.
Running an Aspire Node
AspireWallet-Specific
Getting a SSL Certificate
By default, the system is set up to use a self-signed SSL certificate. If you are hosting your services for others, you should get your own SSL certificate from your DNS registrar so that your users don’t see a certificate warning when they visit your site.
Once you have that certificate, create a nginx-compatible
.pem
file. Copy that.pem
file tofederatednode/config/aspirewallet/ssl/aspirewallet.pem
and the cooresponding certificate.key
file tofederatednode/config/aspirewallet/ssl/aspirewallet.key
. (Note that there will be aaspirewallet.key
andaspirewallet.pem
file already there, which are the default, self-signed certificates, and can be safely overridden.) Then, restart theaspirewallet
service for the new certificate to take effect.
Running an Aspire Node
AspireWallet-Specific
Monitoring the Server
To monitor the server, you can use a 3rd-party service such as Pingdom or StatusCake. The federated node allows these (and any other monitoring service) to query the basic status of the Federated Node via making a HTTP GET call to one of the following URLs:
/_api/
(for mainnet)/_t_api/
(for testnet)
If all services are up, a HTTP 200 response with the following data will be returned:
{"aspire
-server": "OK", "aspireblock_ver": "1.3.0", "aspire-server_ver": "9.31.0", "aspireblock": "OK", "aspireblock_check_elapsed": 0.0039348602294921875, "aspire-server_last_block": { "block_hash": "0000000000000000313c4708da5b676f453b41d566832f80809bc4cb141ab2cd", "block_index": 311234, "block_time": 1405638212}, "local_online_users": 7, "aspire-server_check_elapsed": 0.003687143325805664, "aspireblock_error": null, "aspire-server_last_message_index": 91865}
Note the "aspire-server": "OK"
and "aspireblock": "OK"
items.
If all services but aspire-server
are up, a HTTP 500 response with "aspire-server": "NOT OK"
, for instance.
If aspireblock
is not working properly, nginx
will return a HTTP 503 (Gateway unavailable) or 500 response.
If nginx
is not working properly, either a HTTP 5xx response, or no response at all (i.e. timeout) will be returned.
Running an Aspire Node
AspireWallet-Specific
Creating a configuration file
AspireWallet can be configured via editing the aspirewallet.conf.json
file, via issuing the following command:
sudo docker exec -it federatednode_aspirewallet_1 vim /aspirewallet/aspirewallet.conf.json
This file will contain a valid JSON-formatted object, containing an a number of possible configuration properties. For example::
{ "servers": [ "aspireblock1.mydomain.com", "aspireblock2.mydomain.com", "aspireblock3.mydomain.com" ], "forceTestnet": true, "googleAnalyticsUA": "UA-48454783-2", "googleAnalyticsUA-testnet": "UA-48454783-4", "rollbarAccessToken": "39d23b5a512f4169c98fc922f0d1b121Click to send altcoins to this GASP address ", "disabledFeatures": ["betting"], "restrictedAreas": { "pages/betting.html": ["US"], "pages/openbets.html": ["US"], "pages/matchedbets.html": ["US"], "dividend": ["US"] }, }
Here’s a description of the possible fields:
Required fields:
- servers: AspireWallet should work out-of-the-box in a scenario where you have a single Aspireblock Federated Node that both hosts the static site content, as well as the backend Aspireblock API services. However, Aspirewallet can also be set up to work in MultiAPI mode, where it can query more than one server (to allow for both redundancy and load balancing). To do this, set this
servers
parameter as a list of multiple server URIs. Each URI can have ahttp://
orhttps://
prefix (we strongly recommend using HTTPS), and the strings must not end in a slash (just leave it off). If the server hostname does not start withhttp://
orhttps://
, thenhttps://
is assumed.
If you just want to use the current server (and don’t have a multi-server setup), just specify this as []
(empty list).*
Optional fields:
- forceTestnet: Set to true to always use testnet (not requiring ‘testnet’ in the FQDN, or the ‘?testnet=1’ parameter in the URL.
- googleAnalyticsUA / googleAnalyticsUA-testnet: Set to enable google analytics for mainnet/testnet. You must have a google analytics account.
- rollbarAccessToken: Set to enable client-side error tracking via rollbar.com. Must have a rollbar account.
- disabledFeatures: Set to a list of zero or more features to disable in the UI. Possible features are:
dividend
,exchange
,leaderboard
,portfolio
,stats
andhistory
. Normally this can just be[]
(an empty list) to not disable anything. - restrictedAreas: Set to an object containing a specific page path as the key (or “dividend” for dividend functionality), and a list of one or more ISO 2-letter country codes as the key value, to allow for country-level blacklisting of pages/features.
Once done, save this file and make sure it exists on all servers you are hosting AspireWallet static content on, and restart the aspirewallet
service. Now, when you go to your AspireWallet site, the server will read in this file immediately after loading the page, and set the list of backend API hosts from it automatically.