Welcome, Guest. Please login or register.

Welcome to the StellarVerse Forums!

I have created these forums as a *stable* and *lasting* alternative to other Stellar forums for developers and everybody else! Read more. Questions? PM -> mmathias! Thank you!

Author Topic: Release 0.28, single validator and running public nodes  (Read 5539 times)

Nicolas

  • Stellar Development Foundation
  • Newbie
  • *
  • Posts: 4
    • View Profile
Release 0.28, single validator and running public nodes
« on: December 17, 2014, 09:21:20 PM »
Hello (first post!)

I wanted to keep people updated on what we've been doing to help with the stability issues of the Stellar network.

In the past few days we've completed moving to one validator, that came with a few bonus features.

One interesting thing with the move was that it actually put in focus even more the sync problem that many people have been seeing since the ledger grew to a good size:
Even though the one validator was churning along, public nodes, and in particular the heavily loaded ones (wrt amount of data in the historical node store), were going in and out of sync with the network very often.

Think of this situation with public nodes as runners trying to keep up with leaders (that don't care about the non-leaders) that run faster than them while running inside a maze! A single leader is actually faster as it doesn't spend time talking to other leaders on where to go next.

I thought this would continue to happen and decided to revive some of the changes I had on the consensus front to bias towards safety and help with debugging.

I wrote a doc to help explain the changes I made in the area across the various commits. The current version is https://github.com/stellar/stellard/blob/master/src/ripple_app/consensus/Legacy_consensus.txt

With the changes I made I could isolate where delays were being injected. It gives more leg to the theories I had on potential causes for the quorum to be lost over time (delays range from sub-second to double digit ones).
Quorum loss is a really bad problem to have with multiple validators especially when quorum is re-established between peers that may be different than the ones that had quorum before. In the case of public nodes this translates to availability problems that slow down clients a lot: the node doesn't accept new transactions or accepts transactions that are never validated.

More details on the actual changes can be found at https://github.com/stellar/stellard/pull/184/commits

Those changes worked well with our problematic nodes (those slow nodes stay in sync for much longer period of time: from minutes at a time to hours/days at a time) and I imagine that they should help other public nodes.

You can find updated config files to run your own public nodes in the tree.

Happy holidays,

Nicolas


mmathias

  • Global Moderator
  • Newbie
  • *****
  • Posts: 37
    • View Profile
  • Tech Interests: C++, PHP, Backend
  • Twitter: @mmathias_mmint
Re: Release 0.28, single validator and running public nodes
« Reply #1 on: December 17, 2014, 10:22:04 PM »
Hi Nicolas,

thank you for your update (and of course for your hard work)!

Are there any plans to get rid of some of stellard's dependencies? I actually thought about running a public node, but compiling stellard just seems to be a bit of a hurdle. Are there any plans for stellard to be integrated into Linux or BSD packages and/or ports repositories? Would that even make sense given the changes that will be implemented into the consensus algorithm?

Best,

- Mathias
¯\_(ツ)_/¯
Support StellarVerse: gLVXANgQtoNPaK9Nr4egdvh36jqJ9LMG1A

Nicolas

  • Stellar Development Foundation
  • Newbie
  • *
  • Posts: 4
    • View Profile
Re: Release 0.28, single validator and running public nodes
« Reply #2 on: December 18, 2014, 12:13:28 AM »
Concerning ease of building from source, yes we're working on that.
I don't think we talked about maintaining distro specific collaterals, I imagine that it will be next after making build push button.

Nicolas


xekoukou

  • Newbie
  • *
  • Posts: 8
    • View Profile
  • Tech Interests: Reactive model-oriented programming & mathematics
Re: Release 0.28, single validator and running public nodes
« Reply #3 on: December 19, 2014, 01:33:00 AM »
Hello all,

There are 2 processes that are done by public nodes:

a) Transmitting new transactions.
b) Keeping in sync with the ledger.

In my opinion, those can be separated.

In the first case, we need the server to transmit transactions and receive confirmations from a number of validators and he believes that the transaction has been processed if 81% have returned true.

Keeping in sync with the ledger alone doesn't require that it is current. There can be a delay if need be. This process is simply to keep track of the history of transactions.

The reason that a) doesn't require b) is that if our topology is bad and the percentage of bad actors is bigger than what our protocol can handle, then those factors will continue to exist in all the subsequent transactions. I can't see why b) would help a).

Also, are there any news on the new consensus?

dmathewwws

  • Newbie
  • *
  • Posts: 1
    • View Profile
Re: Release 0.28, single validator and running public nodes
« Reply #4 on: December 20, 2014, 01:49:28 AM »
I actually thought about running a public node, but compiling stellard just seems to be a bit of a hurdle.

You should have a look at docker and https://registry.hub.docker.com/u/stellar/stellard/

It dont think it work until they enable more validators, but I was able to start a stellard server in no time running that dockerfile. Make sure you have enough memory available, I believe it requires 4GB Memory.

Nicolas

  • Stellar Development Foundation
  • Newbie
  • *
  • Posts: 4
    • View Profile
Re: Release 0.28, single validator and running public nodes
« Reply #5 on: January 06, 2015, 06:44:34 AM »
Hello

the reason for not forwarding transactions to the rest of the network when not in sync with the network is to avoid spamming the network with bad transaction when possible (we actually made this a little less strict in stellard compared to rippled):
when in sync, if a transaction "makes sense" against the current ledger, it can be forwarded to the network, otherwise it can just be outright rejected (and no fee is collected).


Nicolas

  • Stellar Development Foundation
  • Newbie
  • *
  • Posts: 4
    • View Profile
Re: Release 0.28, single validator and running public nodes
« Reply #6 on: January 06, 2015, 06:51:42 AM »
Going back on topic, did anybody try 0.28 to see if you can stay in sync?

As a point of reference: for the larger nodes (with 100TB+ rocksdb), we typically stay in sync days at a time, for smaller nodes (with only the last 2-3 weeks worth of ledgers) they have been staying in sync for much longer period of time (order of week+). Original sync can take a few hours, so be patient.
This was not the case before those changes where we would see nodes staying in sync only minutes at a time (for the larger nodes, or even get stuck in "we're not running on the consensus ledger" for days).

To experiment with this, I would start with a very conservative setup (like only download the last 10 ledgers), and progressively increase it (from what I've seen, downloading older ledgers can take its toll on a node that is barely in sync).

Nicolas