This project is based on Bitnodes.

Neurainodes is currently being developed to estimate the size of the Neurai network by finding all the reachable nodes in the network. These are the nodes that accept incoming connections. Why you should run a full node is explained here on the Bitcoin wiki. The current methodology involves sending getaddr messages recursively to find all the reachable nodes in the network, starting from a set of seed nodes. It is worth mentioning that this method of estimating network size does not list all full nodes because not all nodes have an open port that can be probed using Neurainodes. These nodes are either behind firewalls or they are configured to not listen for connections.

The goal is to run the service as efficient as possible. The ravennodes crawler, web_updater, and website currently all run on an a1 medium AWS server (1 vCPu, 2GB RAM).

• Neurainodes
  • TOC
• Neurainodes Crawler
  • Main Changes
  • Dependencies
    • Python Packages
  • Steps on setting up a machine to run Neurainodes
    • Ubuntu 18.04:
      • Install redis
      • Install Neurainodes and set up dependencies
      • Update open file limits to prevent Neurainodes crashing on IO errors
      • Start redis service
      • To start the Neurainodes crawler
• Neurainodes Web-updater
  • Dependencies
    • Python Packages
  • To start the Neurainodes Web-updater
• Neurainodes website
  • Dependencies
    • Python Packages
  • Website hosting
• Call for improvements

• Changed parameters to support Neurai
• Turned off Tor network support.

• Python 2.7 virtual environment
• Redis

sudo apt update && sudo apt upgrade
cd ~/
sudo apt install redis

# I prefer running redis manually
# Optional, remove the service: sudo service redis-server disable

#Move redis conf file
sudo cp [LOCATION]/crawler/depends/redis/redis.conf /etc/redis/
#Install Python requirements
cd [LOCATION]/crawler
pip install -r requirements.txt
#Update GeoIP
bash geoip/update.sh

NOTE: For GeoIP, you now need an account at maxmind.com and acquire a license key. Add this key to ~/.bashrc:

export MAXMIND_LICENSE_KEY=YOUR-KEY-HERE

# Edit the following file:
sudo nano /etc/security/limits.conf
	#Add the following:
	* soft nofile 1000000
	* hard nofile 1000000
 	root soft nofile 1000000
	root hard nofile 1000000
	
# If you run an Ubuntu GUI, Edit the following files too:
sudo nano /etc/systemd/user.conf
	#Change the following setting:
	DefaultLimitNOFILE=1000000
sudo nano /etc/systemd/system.conf 
	#Change the following setting:
	DefaultLimitNOFILE=1000000

#Login as admin
sudo -i
#Update /proc/sys/net/core/somaxconn
sysctl -w net.core.somaxconn=511
#Disable THP
echo never > /sys/kernel/mm/transparent_hugepage/enabled
# Set overcommit_memory to 1
sysctl vm.overcommit_memory=1
#Start Redis Server
redis-server /etc/redis/redis.conf
#Alternatively, run redis in the background with redis-server /etc/redis/redis.conf --daemonize yes

#Make sure the file redis.sock is made in /tmp/

Open a new console, activate the python 2.7 virtual environment and start the crawler:

cd [LOCATION]/Neurainodes
./start.sh

Data output will be available in ~/Neurainodes/data/export/5241564e

Process logs will be available in ~/Neurainodes/log

• Data change watcher adaptation from Michael Cho looking for crawler updates
• Data updater generating ravencoin nodes tables and world maps for the website

• Python 3.7 virtual environment

Open a new console, activate the python 3.7 virtual environment and start the updater:

cd [LOCATION]/web_updater
./start.sh

• Website built using flask

• Python 3.7 virtual environment

See website/requirements.txt

Website is hosted according to this guide

Everyone is welcome to propose changes. Currently, the whole system is set up using python for self-educational purposes. Using an SQL type database, for example, will drastically improve performance.

-Jeroz

https://github.com/jeroz1/Neurainodes https://github.com/RavenCommunity/Neurainodes