Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add exchange rate monitoring #951

Merged
merged 5 commits into from Feb 1, 2019
Merged

Conversation

buck54321
Copy link
Member

The exchanges package contains ExchangeBot, which can be used to monitor DCR-BTC exchanges and BTC-fiat indices. Exchangebot can be disabled completely, or individual exchanges can be disabled.
The package currently has 5 Decred-BTC exchanges and 2 BTC-fiat index sources. The package is completely independent of other DCRData packages, but has been integrated with DCRData to demonstrate how I envision its use.

New API endpoints added for retrieving the Decred exchange rate in one of almost 200 base currencies (indices). Results are JSON-formatted and support indenting. Try

http://localhost:7777/api/exchanges
http://localhost:7777/api/exchanges?indent=1&code=EUR
http://localhost:7777/api/exchanges/codes

On the website side, the price is simply displayed in a chosen default base on the homepage for now.

There is a test implemented which will cycle through all the exchanges about 3 times over 6 minutes.

Resolves #711

I think this PR is GTG, but I'm adding [WIP] for now because it's pretty big and somewhat controversial.

@chappjc
Copy link
Member

chappjc commented Jan 22, 2019

As a note to the peanut gallery, this PR is in line with what @buck54321 and I discussed on how best to implement this without "poisoning" dcrdata.

  • It is not always desirable for dcrdata to be contacting external sources (besides the chain server)
  • dcrdata's no-exchange-monitor and the individual exchange disabling option makes dcrdata's use of this behaviour is entirely optional (since it must be enabled with --exchange-montior).
  • Containing the functionality to its own package opens the door to implementing a separate rate aggregator service.

What is not done in this PR (and that is OK), is developing a stand-alone rate server (e.g. dcrrates), which would:

  1. Aggregate exchange data
  2. Serve the data via HTTP or (g)RPC
  3. Run on an entirely separate machine from dcrdata.
  4. Supply data to 1 or more dcrdata instances.

I feel these features of a "dcrrates" process are important since dcrdata is a backend service that is often run on many instances behind a load balancer. It is not necessary for each instance to be hitting up the exchanges. It is also a security concern having the backends communicating with other sites, not only because dcrdata's outgoing http requests to the exchange APIs are a direct risk to the dcrdata process, but it becomes challenging to conceal identity of the backends (would need NAT, a proxy, etc.).

config.go Outdated Show resolved Hide resolved
config.go Outdated Show resolved Hide resolved
config.go Outdated Show resolved Hide resolved
main.go Show resolved Hide resolved
log.go Show resolved Hide resolved
@chappjc chappjc added this to the 4.0 milestone Jan 29, 2019
@buck54321 buck54321 changed the title [WIP] Add exchange rate monitoring Add exchange rate monitoring Jan 29, 2019
@chappjc
Copy link
Member

chappjc commented Jan 29, 2019

This LGTM. Will leave open and solicit opinions again today.

@xaur
Copy link

xaur commented Jan 29, 2019

Does it overlap with https://github.com/raedahgroup/dcrextdata ?

@chappjc
Copy link
Member

chappjc commented Jan 30, 2019

@xaur, yes and no.

dcrextdata has postgresql storage for a different aim. dcrextdata is a PoC with just poloniex data.

dcrdata's exchange package (this PR) has a more formal and generalized architecture with type Exchange interface, which is currently implemented for 7 different exchanges.

They are similar in ways, but a new package for dcrdata was the best choice IMO.

`exchanges` package contains `ExchangeBot`, which can be used to
monitor DCR-BTC exchanges and BTC-fiat indices. Exchangebot can be
disabled completely, or individual exchanges can be disabled.
Currently has 5 Decred-BTC exchanges and 2 BTC-fiat index sources.

New API endpoint added for retreiving the Decred exchange data in
one of almost 200 base currencies (indices). Results are JSON
formatted and support indenting.

Initially, the price is simply displayed in a chosen default base
on the homepage.
Request time is logged before request is sent to prevent multiple
requests while the connection times out. Defaults for dcrdata
disable huobi and dragonex. ExchangeBot is disabled by default.

Added API data to the README.
@chappjc
Copy link
Member

chappjc commented Feb 1, 2019

I was getting ready to merge this, but I see there are conflicts in home.tmpl from the recent design work you and gozart were doing. There's also a conflict explorerroutes.go, but it's trivial.

Draft merge commit message:

This adds exchange rate monitoring and a new exchanges package.

The exchanges package contains ExchangeBot, which can be used to
monitor DCR-BTC exchanges and BTC-fiat indices. ExchangeBot must
be explicitly enabled, and individual exchanges can be disabled. Both
huobi and dragonex are disabled by default.
Currently it has 5 DCR-BTC exchanges and 2 BTC-fiat index sources.

A new API endpoint (/api/exchange) is added for retrieving the Decred
exchange data in one of almost 200 base currencies (indices). Results
are JSON-formatted and support indenting.

Initially, the price is simply displayed in a chosen default base on the
homepage.

Request time is logged before request is sent to prevent multiple
requests while the connection times out.

@chappjc
Copy link
Member

chappjc commented Feb 1, 2019

Ah, never mind. My local branch didn't have your force-push. All good now.

@chappjc chappjc merged commit 1c581ad into decred:master Feb 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants