Run Your Own Node

Why Run a Node?

Quick Start

1. Install Node.js

   Download and install Node.js 18 or later from nodejs.org

2. Download OWD

   Clone or download the repository from GitHub

   git clone https://github.com/idev-games/the-open-web-directory.git cd the-open-web-directory
3. Start Your Node

   Run the following command:

   npm start
4. Access Your Node

   Open your browser to http://localhost:80

System Requirements

• Operating System: Windows, macOS, or Linux
• Node.js: Version 18 or later
• Disk Space: 1 GB minimum (configurable up to terabytes)
• RAM: 512 MB minimum (1-2 GB recommended)
• Network: Always-on internet connection (optional but recommended)

Configuration

Customize your node by editing src/config.js:

Storage Contribution

Set how much disk space you want to contribute:

Crawler Settings

Adjust crawling speed and behavior:

Option 1: Join the Main Network

By default, your node connects to the main OWD network via the gateway. This is configured in src/config.js:

Option 2: Create Your Own Private Network

To start a completely independent network (isolated from the main OWD network):

1. Edit Network Configuration

   Open src/config.js and customize your network identity:

   network: { name: 'My Private Network', // Give your network a name } node: { name: 'My First Node', // Name this specific node version: '0.1.0', }
2. Remove Gateway Connection

   Comment out or empty the gateway peers to prevent connecting to the main network:

   gateway: { peers: [] // Empty = isolated network }
3. Start Your First Node

   Run your node - it will operate independently:

   npm start
4. Add More Nodes (Optional)

   To add more nodes to your private network, configure them to connect to your first node. In the additional nodes' src/config.js:

   gateway: { peers: [ { host: 'your-first-node-ip-or-domain', port: 80, // or whatever port you configured name: 'My First Node' } ] }

Running 24/7

For maximum network contribution, run your node continuously:

• Linux: Use systemd or PM2
• All platforms: Use PM2 process manager
• Docker: Run in a container for easy deployment

Using PM2

Make Your Node Public

To allow others to connect and search through your node:

1. Configure your router to forward port 8471 to your computer
2. Set up a domain name (optional)
3. Use a reverse proxy like Nginx for HTTPS (recommended)
4. Share your node address with the community

Getting Help

• Documentation: README.md
• Issues: GitHub Issues

Technical Details

What Your Node Does

• Crawls websites starting from seed URLs
• Extracts titles, descriptions, and links
• Stores indexed data locally in compact JSONL format
• Shares search results with the network
• Discovers and connects to peer nodes
• Provides HTTP API for search queries

Privacy & Bandwidth

• Your node only crawls public web pages
• No personal data is collected or shared
• Bandwidth usage is configurable (default: 1 request/second)
• Respects robots.txt and crawl delays
• Storage is limited to configured amount

Safety & Security

• No downloads: OWD only fetches HTML metadata, not executable files
• No viruses: The crawler extracts only text (titles, descriptions, links) from web pages
• No malicious code execution: Pages are parsed as text, never executed
• Zero dependencies: No third-party packages means no supply chain attacks
• Open source: Every line of code is auditable on GitHub
• Safe by design: OWD does not execute downloaded web content and only stores extracted metadata.

Contributing

Beyond running a node, you can also:

• Report bugs and suggest features
• Contribute code improvements
• Improve documentation
• Share OWD with others

See CONTRIBUTING.md for details.