Logged Conversation (all times UTC)
[16:59] <stellar-slack> So, since I’m rewriting horizon in go, and am not bound by rails’ conventions, should I bring horizon’s configuration mechanism into line with stellar-core? i.e. a TOML config file specified on the command line. Thoughts?
[17:01] <stellar-slack> @scott depends how much configuration is required. The flag package works a treat for few parameters with sensible defaults:
[17:01] <stellar-slack> https://golang.org/pkg/flag/
[17:01] <stellar-slack> What do you need to configure apart from the SQL connection string?
[17:06] <stellar-slack> At present, the configurable options will be:
[17:06] <stellar-slack> - stellar core db url
[17:06] <stellar-slack> - horizon db url
[17:06] <stellar-slack> - memcache hosts
[17:06] <stellar-slack> - friendbot secret seed
[17:06] <stellar-slack> - rate limiting configuration
[17:06] <stellar-slack> I’m sure there will be more
[17:06] <stellar-slack> possibly https://github.com/spf13/viper ?
[17:06] <stellar-slack> oh interesting
[17:07] <stellar-slack> just googling around. I'm not super into the go ecosystem but a few people suggest that and it looks nice ish
[17:08] <stellar-slack> (relative of https://github.com/spf13/cobra seemingly)
[17:09] <stellar-slack> Call me minimalist, but you could probably achieve the given list with 5 lines of the flag package :simple_smile:
[17:09] <stellar-slack> scott: I didn't quite get the rewrite-in-go discussion; I am not the least opposed to it, merely wonder what the falling-down part was on using your previous choice .. jruby + something?
[17:20] <stellar-slack> graydon:
[17:20] <stellar-slack> Basically, we evaluated several choices for horizon and came up with a prioritized list that looked like:
[17:20] <stellar-slack> - jruby + rails
[17:20] <stellar-slack> - go
[17:20] <stellar-slack> - node.js
[17:20] <stellar-slack> - everything else (clojure, http://asp.net|asp.net, python)
[17:20] <stellar-slack> jruby-rack was chosen because it would a) leverage my existing experience with rails to let us concentrate on API design and not "How do I do this in language X", b) the deployment story is better than plain rails and c) performance would be acceptable compared to MRI rails.
[17:20] <stellar-slack> Unfortunately, jruby-rack, which lets you run your rails app on a java application server, does not have support for asynchronous responses... it will buffer until your close the response and it does not support multiple concurrent requests. Not realizing that deficiency is completely my fault. I should have proved out that capability sooner. The async and streaming responses for horizon are going to be pivotal for
[17:21] <stellar-slack> I think it's that point I didn't get; what do you mean by async responses?
[17:22] <stellar-slack> I mean HTTP has a lot of semantics, I'm curious which one we're intending to lean on and why. apologies if this is a sort of pointless digression, it just caught my ear and seems like it's causing you a major course-correction so I'm curious
[17:23] <stellar-slack> the old sever did websockets which are entirely unlike http request/response. but you're talking about something else, like multiple response chunks or something?
[17:23] <stellar-slack> So, we’ll be leveraging long-polling to wait on transaction settlement from horizon for ease of use. And we’ll be using Server Sent Events to let people follow along with events as ledgers close on the network
[17:23] <stellar-slack> yeah, chunked responses
[17:27] <stellar-slack> Take for example the desired integration: “I want to follow along with every ledger close event"
[17:28] <stellar-slack> With the websocket api from stellard you would have to write a bit of logic to deal with the fact that you may get a dropped connection and miss a ledger close
[17:28] <stellar-slack> You then have to use a different method to load that ledger and process it
[17:29] <stellar-slack> With our new design, you are always including a token that represents the last ledger you’ve seen, so we can serve you an unbroken chain of ledgers from the same endpoint
[17:29] <stellar-slack> IMO, it makes for a simpler integration
[17:30] <stellar-slack> ok
[17:30] <stellar-slack> For the record though, there’s nothing stopping you from doing that with websockets (and we will most likely offer that as well)
[17:30] <stellar-slack> *shrug* I am not well enough versed to be able to trade off the tradeoffs; I was just trying to understand what blocked you
[17:31] <stellar-slack> makes sense now, thanks
[17:31] <stellar-slack> anytime
[17:31] <stellar-slack> Go’s http package has nice support for chunking responses:
[17:31] <stellar-slack> http://golang.org/src/net/http/server.go?s=2698:2782#L221
[17:31] <stellar-slack> http://golang.org/pkg/net/http/#Flusher
[17:31] <stellar-slack> Which works very well with SSE servers:
[17:31] <stellar-slack> https://github.com/donovanhide/eventsource/blob/master/server.go#L78-L87
[17:31] <stellar-slack> It’s not really a language problem, just more that Go has a great standard library for doing interesting things with HTTP :simple_smile:
[17:33] <stellar-slack> yeah, definitely.
[17:33] <stellar-slack> (https://github.com/jruby/jruby-rack .. *seems* to talk about chunking responses, and gives you access to the raw java servlet response object, but this is really way out of my depth...)
[17:34] <stellar-slack> This is the blocker: https://github.com/jruby/jruby-rack/issues/119
[17:35] <stellar-slack> rats!
[17:35] <stellar-slack> donovan: I gather sse isn't supported in ie yet?
[17:36] <stellar-slack> there’s a couple of polyfills for it, but not natively
[17:37] <stellar-slack> SSE is a HTTP response that doesn’t get closed :simple_smile:
[17:37] <stellar-slack> It’s also one way, unlike websockets.
[17:38] <stellar-slack> mm
[17:39] <stellar-slack> Main selling point for browsers is that you don’t have to write code to handle disconnection, the browser does it all for you.
[17:40] <stellar-slack> This site is a nice demo:
[17:40] <stellar-slack> https://bitcoinity.org/markets
[17:40] <stellar-slack> oh wow, it's really simple on the client side
[17:41] <stellar-slack> ```curl https://bitcoinity.org/ev/markets/```
[17:41] <stellar-slack> works nice for other clients too :simple_smile:
[17:41] <stellar-slack> right, ok. quasi-competing tech but you probably want to use sse when you can then.
[17:42] <stellar-slack> interesting
[17:42] <stellar-slack> It’s basically as simple as it is possible to make streaming. Hidden gem of HTML 5.
[17:43] <stellar-slack> yeah, I’ve got nothing but good things to say about it.
[17:44] <stellar-slack> well, except that it's always the last thing that gets implemented in a server-side framework for some reason :confused:
[17:52] <stellar-slack> ^ wow, @donovan that was really cool
[17:54] <stellar-slack> reason it's maybe last at this point is because most scenarios waiting for notifications are much longer than this (news feed, etc) and on mobile you should not call such APIs: apps should use OS level notifications to avoid many hanging connections that kills battery life.
[17:58] <stellar-slack> @monsieurnicolas: yes, but that’s an implementation detail of the mobile browser, not the app running in the browser. It’s covered here:
[17:58] <stellar-slack> http://www.w3.org/TR/2012/WD-eventsource-20120426/#eventsource-push
[17:58] <stellar-slack> I doubt mobile browsers do a better job at saving batteries when websockets are running.
[18:00] <stellar-slack> oh I didn't know SSE had this level of integration - so yes, then it's much superior to websockets in that respect
[18:00] <stellar-slack> I think the trick is to define endpoints which only send the data relevant to the client, which will be much more intermittent. In other words, if I’m interesting in only transactions affecting my account, I don’t want every other single transaction that is on the network to be sent to me.
[18:01] <stellar-slack> I may have just described another system above :simple_smile:
[18:07] <stellar-slack> yes, I guess it's not because it's in the RFC that devices actually automagically take care of rerouting to the appropriate notification service :simple_smile: Given that this doc is from Google, I imagine it's basically what is done for Android
[20:19] <stellar-slack1> The beginnings: https://github.com/stellar/go-horizon
[20:19] <stellar-slack1> temporary home while we work to reach parity with stellar/horizon
[20:31] <stellar-slack1> :clap: :confetti_ball:
[21:33] <stellar-slack1> @matschaffer: figured out why I wasn’t getting any coverage reporting from goconvey… the homebrew package for go doesn’t install any additional tools. I had to `go install http://golang.org/x/tools/cmd/cover|golang.org/x/tools/cmd/cover`, then it all works
[21:34] <stellar-slack1> weird. Don’t think I had to do that (doesn’t show in my history). Are you on the latest golang?
[21:34] <stellar-slack1> sorta wonder if maybe I’m on a slightly newer pkg that ships with the cover tool
[21:35] <stellar-slack1> I’m on 1.4.2 installed from the pkg distro
[21:35] <stellar-slack1> ah, I bet the .pkg includes them
[21:35] <stellar-slack1> I’m using homebrew
[21:35] <stellar-slack1> ah, right. that just clicked for me. So guess brew ships a more trimmed set or something
[21:36] <stellar-slack1> yep, I saw some blog post that explained it (The trigger was that I noticed I didn’t have `godoc` either)
[21:39] <stellar-slack1> ah. That’s kinda lame
[21:39] <stellar-slack1> I guess they have their reasons though
[21:41] <stellar-slack1> godoc was removed from the main install in 1.2:
[21:41] <stellar-slack1> https://golang.org/doc/go1.2#go_doc
[21:41] <stellar-slack1> and the cover tool was never part of the main install but did move repo in 1.4
[21:41] <stellar-slack1> https://golang.org/doc/go1.4#subrepo
[22:38] <stellar-slack1> ah, so guessing brew just picks up the main install whereas the pkg includes the subrepo tools
[23:09] <stellar-slack1> @matschaffer: no, vanilla Go install on brew and via download gives:
[23:09] <stellar-slack1> ```$ go tool cover
[23:09] <stellar-slack1> go tool: no such tool "cover"; to install:
[23:09] <stellar-slack1> go get http://golang.org/x/tools/cmd/cover|golang.org/x/tools/cmd/cover```
[23:09] <stellar-slack1> thanks! good to know
[23:10] <stellar-slack1> `godoc` is included, `go doc` got renamed :simple_smile:
About StellarVerse IRC Logger
StellarValue IRC Logger
is part of