I support this proposal.
Release of new/updated pool software
Snipa22 posted this 7 years ago
What
An updated pool for usage within the XMR community in order to help continue growing smaller pools within the Monero Community. This project was originally spawned off of the project by Wolf0 & OhGodAGirl to replace and improve the XMR Stratum, but provides a baseline to work from for more modern implementations of the pool software.
This also includes many features that are valuable to miners at this time, including support for Integrated Addresses, Payment ID Based payouts, Per-Worker Monitoring, and Per-Worker E-mail Notifications.
Who
I'm Snipa on IRC, I've been a professional Linux sysadmin for the past 10-ish years, developing python based software for the past 4-5, and work primarily as a DevOps engineer in my full time position, maintaining servers and backend code, while providing project guidance for our main workflow at my FT position.
This has been my first serious foray into the XMR community, as well, into NodeJS, and while I've stumbled quite a bit, I feel like this is a product I am pleased to release to the community to play with and see how they like it as well.
I am a regular in the #monero-pools channel, and have provided assistance with things such as interim-pool fixes for the 127 transaction bug, information about the Claymore SSL implementation, and the certs (Unsigned certs ftw!) required for this implementation. I've also provided assistance to new pool ops looking to get up and going. I thoroughly believe that XMR is stronger for the small pool ops, and it's my goal to get this code out to make it easier and simpler to setup. I've also provided first-pass documentation on the current pool API's, as they're undocumented: https://cryptonote-xmr-pool-api.restlet.io
Why
This was spawned as a pool target to replace the issues with Proportional payouts, such as pool hoppers, as used by Clintar and Zone117x's pools, allowing for better payouts for miners who are interested in long-term profit. This was also to be the target for the new Stratum Protocol as to be developed by Wolf0 & OhGodAGirl. This code leverages many stable NodeJS libraries, fixes to versions, and uses the current 6.x.x LTS release. This also offers a number of new ways to setup mining pools, including PPS and Solo mining, for miners who like to know exactly what they'll get paid, and miners who want to gamble and roll the dice while mining.
The Proposal/Milestones
The bulk of this code is complete at this time, and is running a live pool at https://xmrpool.net, which serves as the living, breathing example of what this pool is able to accomplish.
Milestone 1
Target: Release of the pool source in it's current form, including SQL schemas and all things required to setup and run the pool with minimal documentation.
At this point, I've put approximately 40-60 hours of work into the development of this pool, lost a small chunk of XMR to double-payout bugs (Which have been fixed), ordered approximately 1 BTC worth of NH hash time to stress test, load-test, and verify functionality of the pool on the production network. The API is decently well documented at: https://cryptonote-pool-api.restlet.io/ and will be released in full Swagger format for anyone looking to integrate with the pool API.
As such, I'm setting a target of 600 XMR for this milestone, while high, this has been a very time consuming project. This is also conditionally to assist in paying out Mesh00 for his work on the frontend, and Wolf0 & OhGodAGirl to release the AES-NI sped-up version of the multi-hashing library that this pool takes full advantage of, allowing it to run on much smaller servers (VPS) while still doing extensive share verification and validation.
Milestone 2
Target: Finishing admin-related APIs and Frontend, moving to Websockets for statistical information and frontend data pushes, rather than pulls
At this time, the pool does not provide a dedicated admin interface or APIs, I have found these to be largely not-needed, as I am a full time system administrator. Overall, these should be fairly simple, as they're mostly accessing data that's in SQL anyway, as well as verifing wallet-state, etc. Adding websockets is non-trivial to a project of this nature, as the current architecture is not specifically designed to accept websockets, however, due to the use of LMDB and it's zero-copy systems, all of the core daemons share a universal cache that can be used readily to deal with passing data back and forth between applications.
As such, I am setting a target of 150 XMR for this milestone, to fund both frontend and backend work on this project.
Milestone 3
Target: Documenting and polishing the installation process
While not complex, and I have scripts for the deployment of this system in my multi-server environment, this would allow a new user to execute a single script to bring online a pool with a minimum of questions being asked. The goal here is to allow small pool ops to bring online pools more quickly. This is not required, and installation support will always be provided in #monero-pools
As such, I am setting a target of 50 XMR for this milestone.
Closing
This FFS is mostly to help me recoup the time, funding provided into this project, and provide funding to those who have opted to help through this process.
Thanks guys! I'm glad to see the support. Code's coming along more rapidly in the background, just fixed a couple of the auto-emailing bugs, and shifted more stuff into SQL for easy configuration. As soon as we hit milestone 2, all of the API's will be in place, and hopefully, the frontend for the admin panel will be done pretty quickly as well. Just working on polishing the API's right now. It's coming along great everyone!
Hello All,
It's been an interesting couple of weeks, and I'd like to thank several people for their work in improving the pool:
FireiceUK for tracking down an SSL bug with the implementation of the pool, which has been patched.
MineCoins for requesting MinerGate support and updating the deployment scripts
Didrole for fixing the XMR proxy support
We've hit Milestone 2 and 3 at this time, Socket.IO support has been added to the pool for all major/heavy/recurring frontend APIs, documentation at: https://github.com/Snipa22/nodejs-pool/wiki/socket.io Deployment scripts have been tweaked and tuned at this time as well. The Admin frontend is coming along nicely, and is currently read-only, with writing coming down the wire from Mesh00.
We've added a small army of new features as well!
# Minergate passwordless logins.
# Nicehash auto-targeting to 100k diff
# Socket.io
# XMR Proxy support
# Debug and management scripts
# Fixed deployment on stock MySQL servers
# Keepalive support
# Valid/Invalid share tracking
We're not done yet either, feel free to post on the pool's repo if you have requests for new things to add!
will support this project.
That last 6 was bugging me. Looking forward to the results, appreciate the work so far :)
Thanks for the support everyone! I'll be aiming to release the current version this weekend, along with a likely beta form of the transforming scripts (to move from the old pools to this one!) and the installation scripts, knocking out Milestone 1/3. 2 is on it's way, right now, there's still some SQL work that needs to be done by hand (Trying to knock it out, promise, all the API's are in, just waiting for the frontend to catch up), and there's some fantastic new frontend stuff coming down the pipe (You should see it, it's shiny!)
I'm happy to announce that the following git repos are now available for cloning:
Pool: https://github.com/Snipa22/nodejs-pool
Pool Site: https://github.com/hackfanatic/xmrpoolui
Node MultiHashing w/ AES-NI: https://github.com/Snipa22/node-multi-hashing-aesni
Node Cryptonote Util w/ Fix for Integrated Addresses: https://github.com/Snipa22/node-cryptonote-util/tree/xmr-Nan-2.0
Many thanks to the following persons:
Wolf0/OhGodAGirl for Node MultiHashing having AES-NI added.
Mesh00 for the frontend work in AngularJS
Zone117x for the original node-cryptonote-pool implementation that was the basis for some of this work.
Updates made from the last post:
The installer scripts have been tested on a 4Gb Linode server, and can perform a clean install in about an hour, spending the majority of it's time building the Monero Daemon, as well as downloading and initing the block chain.
Difficulty calculations should be less painful now, as we're now calculating a miners H/S based on the number of hashes submitted from connection, then using the pool's time multiplier to get it under control.
There's several new API endpoints that are pending update in the documentation, to bulk-grab worker stats as well as worker hashrate "charts" which are lists of hashes/second.
Major things still pending:
Mesh00 and I are going to be adding socket.io support to the API server and frontend to allow pubsub based systems. Administrative API's are complete, and just need to be added to the frontend.
Sneak peeks of the frontend are available on Mesh00's repo, under the material branch!
With the repos now open-sourced, including work from Wolf0/OhGodAGirl for the multihashing, and the pool site from Mesh00, I'm happy to announce that Milestone 1 has been hit! We've got several (3-5 that I'm aware of) pool ops who have begun to look at moving from the current node-cryptonote-pool, and several who have deployed. My thanks go out to these people for getting us an extra round of testing.
Milestone 2 is fast approaching, with all of the Admin API's in place, and the frontend coming together! We'll be going forwards with Socket.io and will be documenting this along the way.
Milestone 3 will likely finish up quickly after Milestone 2, as we have the base installer scripts in place (Continually getting patched with new bugs!)
Changes from the last post:
Pool has formally been pushed to the MIT license. We welcome code updates from downstream, and even encourage it!
Changed the default claymore port to 9000 to get it out of the way of the remote share daemon
Updated PM2 launches to timestamp log files.
Updated the README with additional instructions, and the wiki with details of some of the more complex configuration options.
Bunch of SQL fixes and updates to the blockManager to get it to play more nicely and not re-queue work that are already being worked on.
Things coming down the line:
E-mail templates - Moving away from the hard-coded emails in the system and into templates stored in SQL. This will make it much easier for pool ops to customize their emails.
Install script version locking - The install script is build for 16.04 LTS. Nothing else. We need to lock this out.
Digest Wallet Authentication - With the current Master branch of Monero, the RPC wallet now offers digest authentication, so we'll be moving towards that!
Things to discuss:
Offering PPLNS as a time-based lockout, rather than N Shares AKA PPLNH/PPLNM: https://github.com/Snipa22/nodejs-pool/issues/12
If you have suggestions or ideas, please feel free to pop open a issue on the pool's github!
I support this proposal and can vouch for Snipa's awesomeness. I've trusted him enough to merge my old moneroworld.com pool into xmrpool.net. In addition to the pool server work, he also runs a high speed relay network at node.xmrbackb.one that he makes available for remote nodes, and is hosting explore.moneroworld.com.
Long story short, he's totally into Monero and can do great work.
And our pool network needs all the diversity it can get! And we'll finally have a PPLNS system. No more pool hoppers!
I'm in! But... How does funding work? Will there be a special wallet for donations?
You simply hit the contribute button to get the info!
Just a quick update, I was about half asleep when I wrote this, and neglected to credit OhGodAGirl, who has/will be working with Wolf0 on their portions of this project! I'd normally edit this into the body of the message, but it looks like that might break the FFS system.