Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exchange templates update #2

Open
wants to merge 114 commits into
base: exch_tmpl_fix
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
00f1bee
build(deps): Bump bufbuild/buf-setup-action from 1.29.0 to 1.30.0 (#1…
dependabot[bot] Mar 11, 2024
13ed40a
build(deps): Bump golang.org/x/net from 0.21.0 to 0.22.0 (#1497)
dependabot[bot] Mar 11, 2024
d679a76
build(deps): Bump google.golang.org/grpc from 1.62.0 to 1.62.1 (#1500)
dependabot[bot] Mar 11, 2024
9d1476d
Coinut: Fix websocket not parsing instruments, improve nonce matching…
gbjk Mar 15, 2024
3e4b9be
Tests: Abstract UpdatePairsOnce (#1503)
gbjk Mar 15, 2024
044f2cd
build(deps): Bump google.golang.org/protobuf from 1.32.0 to 1.33.0 an…
dependabot[bot] Mar 15, 2024
3aad665
common/timedmutex: clean and improve (#1486)
shazbert Mar 20, 2024
05dec88
request: fix intermittent race when verbose and reading the body in d…
shazbert Mar 25, 2024
b115b2b
minor exchange template update
samuael Mar 26, 2024
e64ae62
Merge branch 'master' of http://github.com/thrasher-corp/gocryptotrad…
samuael Mar 26, 2024
917dd6b
Minor fix to test_files template
samuael Mar 26, 2024
9d7bc3e
build(deps): Bump golang.org/x/net from 0.22.0 to 0.24.0 (#1512)
dependabot[bot] Apr 8, 2024
5b2251b
build(deps): Bump bufbuild/buf-setup-action from 1.30.0 to 1.30.1 (#1…
dependabot[bot] Apr 8, 2024
98eae1e
build(deps): Bump google.golang.org/grpc from 1.62.1 to 1.63.0 (#1513)
dependabot[bot] Apr 10, 2024
9657a57
exchanges: shift GetDefaultConfig wrapper function to exchange.go (#1…
shazbert Apr 12, 2024
e823f9e
request/nonce: Refactor to simplify package and prevent consecutive m…
shazbert Apr 12, 2024
b909e15
Bithumb: Enhance test coverage (#1509)
Beadko Apr 15, 2024
6fc6dcd
build(deps): Bump github.com/shopspring/decimal from 1.3.1 to 1.4.0 (…
dependabot[bot] Apr 15, 2024
07af828
build(deps): Bump google.golang.org/grpc from 1.63.0 to 1.63.2 (#1517)
dependabot[bot] Apr 15, 2024
c967d8a
Kraken: Fix wsCancelOrders not erroring with order id (#1505)
gbjk Apr 17, 2024
7b842c2
go: Fix darwin/arm64: toolchain not available version directive (#1523)
gemscng Apr 19, 2024
fdf6014
Currency: Remove Pair Index formatting/parsing (#1520)
gbjk Apr 19, 2024
44d50a3
Kraken: Fix GetLatestFundingRates requiring pair enabled (#1522)
gbjk Apr 19, 2024
694f2f7
gateio: fix spot/futures order issues (#1524)
shazbert Apr 29, 2024
7bad4f7
build(deps): Bump bufbuild/buf-setup-action from 1.30.1 to 1.31.0 (#1…
dependabot[bot] Apr 29, 2024
b46b793
build(deps): Bump github.com/urfave/cli/v2 from 2.27.1 to 2.27.2 (#1527)
dependabot[bot] Apr 29, 2024
1e95ae9
qa: Enforce JSON standard for GCT configs (#1526)
thrasher- Apr 30, 2024
0658b27
build(deps): Bump golangci/golangci-lint-action from 4 to 5 (#1529)
dependabot[bot] May 1, 2024
0676c78
btcmarkets: remove trailing slash on markets/ticker strings and updat…
shazbert May 3, 2024
f1ff951
engine/exchanges: Add GetCurrencyTradeURL wrapper func/gRPC endpoint …
gloriousCode May 3, 2024
a46622a
Bitstamp: Add missing API fields and enhance test coverage (#1510)
Beadko May 6, 2024
da98bd4
build(deps): Bump golang.org/x/text from 0.14.0 to 0.15.0 (#1535)
dependabot[bot] May 6, 2024
3ee65b9
build(deps): Bump google.golang.org/protobuf from 1.33.0 to 1.34.0 (#…
dependabot[bot] May 6, 2024
ece58eb
Bybit: Add support for SOL options (#1533)
gloriousCode May 6, 2024
93c2d01
Test: Internal exchange coverage (#1525)
gbjk May 7, 2024
095394f
build(deps): Bump ikalnytskyi/action-setup-postgres from 5 to 6 (#1537)
dependabot[bot] May 7, 2024
b306095
build(deps): Bump google.golang.org/protobuf from 1.34.0 to 1.34.1 (#…
dependabot[bot] May 13, 2024
b2d7bee
build(deps): Bump golang.org/x/net from 0.24.0 to 0.25.0 (#1540)
dependabot[bot] May 13, 2024
2a92878
build(deps): Bump golangci/golangci-lint-action from 5 to 6 (#1543)
dependabot[bot] May 13, 2024
4cd4fb0
orderbook: Refactor package structure for simplicity and efficiency (…
shazbert May 14, 2024
34ef09d
bugfixes: Backtester credentials, Binance ExecutionLimits, Mock recor…
gloriousCode May 15, 2024
7d1eecf
Tests: Various race fixes and move TestFixtureToDataHandler (#1534)
gbjk May 16, 2024
ca9efe7
currency/manager: remove deadlock potential in concurrent operations …
shazbert May 17, 2024
bd33fc4
build(deps): Bump github.com/grpc-ecosystem/grpc-gateway/v2 (#1547)
dependabot[bot] May 20, 2024
3ae0ecf
build(deps): Bump bufbuild/buf-setup-action from 1.31.0 to 1.32.0 (#1…
dependabot[bot] May 20, 2024
d6bace3
build(deps): Bump bufbuild/buf-setup-action from 1.32.0 to 1.32.1 (#1…
dependabot[bot] May 27, 2024
c37a115
Linter: Spelling fixes (#1554)
gbjk May 28, 2024
aeb4a87
exchanges: Add Deribit exchange support (#1082)
samuael May 29, 2024
f6a95da
exchanges/request: abstract and consolidate rate limiting code to req…
shazbert Jun 3, 2024
abba66f
build(deps): Bump bufbuild/buf-setup-action from 1.32.1 to 1.32.2 (#1…
dependabot[bot] Jun 3, 2024
afb6f75
build(deps): Bump github.com/spf13/viper from 1.18.2 to 1.19.0 (#1560)
dependabot[bot] Jun 3, 2024
1199f38
subscriptions: Encapsulate, replace Pair with Pairs and refactor; imp…
gbjk Jun 7, 2024
e16e16b
Deribit: bug fixes, test fixes and implement GetCurrencyTradeURL (#1558)
gloriousCode Jun 7, 2024
eacf14c
build(deps): Bump golang.org/x/net from 0.25.0 to 0.26.0 (#1562)
dependabot[bot] Jun 10, 2024
06b9980
build(deps): Bump github.com/gorilla/websocket from 1.5.1 to 1.5.2 (#…
dependabot[bot] Jun 10, 2024
4c4b693
gateio: update FuturesAccount and Position structs; rename gateioTime…
shazbert Jun 14, 2024
98f025e
BTSE: Various fixes (#1550)
gbjk Jun 14, 2024
5f05907
build(deps): Bump google.golang.org/protobuf from 1.34.1 to 1.34.2 (#…
dependabot[bot] Jun 17, 2024
2cf2ead
build(deps): Bump bufbuild/buf-setup-action from 1.32.2 to 1.33.0 and…
dependabot[bot] Jun 18, 2024
5946933
build(deps): Bump github.com/gorilla/websocket from 1.5.2 to 1.5.3 (#…
dependabot[bot] Jun 20, 2024
258b60c
build(deps): Bump bufbuild/buf-setup-action from 1.33.0 to 1.34.0 (#1…
dependabot[bot] Jul 1, 2024
b7a2f61
Bybit: Fix WS ticker processing (#1538)
gbjk Jul 3, 2024
48349bc
protocol/order: adds additional fields for trading requirements (#1552)
shazbert Jul 3, 2024
03c1fb7
Bybit: Add missing field min notional (#1574)
shazbert Jul 4, 2024
d4c3a66
build(deps): Bump golang.org/x/net from 0.26.0 to 0.27.0 (#1577)
dependabot[bot] Jul 8, 2024
00c5c95
build(deps): Bump google.golang.org/grpc from 1.64.0 to 1.65.0 (#1576)
dependabot[bot] Jul 8, 2024
c601575
subscriptions: Add templating support and integrate with Binance (#1568)
gbjk Jul 9, 2024
90fee94
currency: Make pairs.Add variadic (#1566)
cranktakular Jul 12, 2024
db59b85
log: allow external definition of log handling (#1561)
shazbert Jul 19, 2024
4f1696f
Subscriptions: Fix subscription.Clone Params and Pairs (#1582)
gbjk Jul 19, 2024
34d9920
Okx: Fix TestGetBlockTrade transient fail (#1583)
gbjk Jul 19, 2024
45ef2b1
Gateio: Add unified account API endpoint support (#1573)
shazbert Jul 23, 2024
425ec0c
orders/gateio: Add GetTradeAmount method to order.Submit type (#1584)
shazbert Jul 23, 2024
d1e3669
Subscriptions: Add List.Clone (#1592)
gbjk Jul 26, 2024
ba3dcc8
Bybit: Fix UpdateAccountInfo coin balance issue, improve GetWalletBal…
thrasher- Jul 29, 2024
0c5d049
Kraken: Simplify REST error and result handling (#1591)
thrasher- Jul 29, 2024
d60ea77
build(deps): Bump github.com/urfave/cli/v2 from 2.27.2 to 2.27.3 (#1594)
dependabot[bot] Jul 29, 2024
1db4622
build(deps): Bump bufbuild/buf-setup-action from 1.34.0 to 1.35.1 (#1…
dependabot[bot] Jul 29, 2024
deaee80
build(deps): Bump github.com/grpc-ecosystem/grpc-gateway/v2 (#1595)
dependabot[bot] Jul 29, 2024
7e83f10
build(deps): Bump golang.org/x/time from 0.5.0 to 0.6.0 (#1602)
dependabot[bot] Aug 5, 2024
b41fe27
Kucoin: Add subscription templating and various fixes (#1579)
gbjk Aug 9, 2024
5f44655
build(deps): Bump golang.org/x/net from 0.27.0 to 0.28.0 (#1607)
dependabot[bot] Aug 12, 2024
1957734
build(deps): Bump bufbuild/buf-setup-action from 1.35.1 to 1.36.0 (#1…
dependabot[bot] Aug 12, 2024
b602d54
build(deps): Bump github.com/urfave/cli/v2 from 2.27.3 to 2.27.4 (#1608)
dependabot[bot] Aug 12, 2024
91ff6c5
orderbook/gateio: add field `UpdatePushedAt` and `InsertedAt` for spe…
shazbert Aug 15, 2024
0becfbd
Kraken: Fix TestGetOpenInterest (#1611)
gbjk Aug 16, 2024
facf291
currency/exchanges: Add bespoke exchange translator and pair matching…
shazbert Aug 16, 2024
225429b
CI/build: Update Go version, linters and fix minor issues (#1612)
thrasher- Aug 16, 2024
17c2ef2
stream/match: Reduce complexity and limit locking when match occurs (…
shazbert Aug 19, 2024
d7907c2
build(deps): Bump bufbuild/buf-setup-action from 1.36.0 to 1.37.0 (#1…
dependabot[bot] Aug 19, 2024
03e9809
build(deps): Bump github.com/grpc-ecosystem/grpc-gateway/v2 (#1621)
dependabot[bot] Aug 19, 2024
1cabba7
common/gateio/stream: add thread-safe counter and overide default Gen…
shazbert Aug 24, 2024
74f4df6
okcoin: Remove exchange implementation (#1626)
thrasher- Aug 24, 2024
f3dda71
build(deps): Bump bufbuild/buf-setup-action from 1.37.0 to 1.38.0 (#1…
dependabot[bot] Aug 26, 2024
15e0eed
tests: Fix various issues (#1634)
thrasher- Aug 27, 2024
649f3df
engine: Add overridable exchanges command line flag (#1630)
thrasher- Aug 27, 2024
2605eab
build(deps): Bump google.golang.org/grpc from 1.65.0 to 1.66.0 (#1637)
dependabot[bot] Sep 2, 2024
7c9e651
build(deps): Bump bufbuild/buf-setup-action from 1.38.0 to 1.39.0 (#1…
dependabot[bot] Sep 2, 2024
cb6b342
exchanges/websocket: update websocket rate limiting to use requester …
shazbert Sep 2, 2024
360afa5
build(deps): Bump golang.org/x/net from 0.28.0 to 0.29.0 (#1643)
dependabot[bot] Sep 9, 2024
4a4d666
build(deps): Bump bufbuild/buf-setup-action from 1.39.0 to 1.40.1 (#1…
dependabot[bot] Sep 9, 2024
a82f0bf
build(deps): Bump github.com/mattn/go-sqlite3 from 1.14.22 to 1.14.23…
dependabot[bot] Sep 9, 2024
46f3080
gateio: fix and optimise time parsing (#1647)
shazbert Sep 12, 2024
22cb0eb
stream: Add verbosity to outbound requests that also returns response…
shazbert Sep 12, 2024
b8e836d
common: Replace StringDataCompare with slices.Contains and cleanup st…
thrasher- Sep 13, 2024
b461c32
stream/okx: allow rate limit definitions to be used by the stream pac…
shazbert Sep 13, 2024
d94b8af
exchanges: Kucoin Update (#1438)
samuael Sep 13, 2024
3ebd2da
CoinbasePro: Skip specific tests due to API deprecation (#1649)
thrasher- Sep 16, 2024
5e663b0
build(deps): Bump bufbuild/buf-setup-action from 1.40.1 to 1.41.0 (#1…
dependabot[bot] Sep 16, 2024
aef8a1f
build(deps): Bump google.golang.org/grpc from 1.66.0 to 1.66.2 (#1651)
dependabot[bot] Sep 16, 2024
c96c0b0
Binance: Fix occassional race failure on TestWsOrderExecutionReport (…
gbjk Sep 19, 2024
007f671
build(deps): Bump bufbuild/buf-setup-action from 1.41.0 to 1.42.0 (#1…
dependabot[bot] Sep 23, 2024
a09fefe
build(deps): Bump google.golang.org/grpc from 1.66.2 to 1.67.0 (#1657)
dependabot[bot] Sep 23, 2024
35b90d1
Merge branch 'master' of http://github.com/thrasher-corp/gocryptotrad…
samuael Sep 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 6 additions & 8 deletions .github/workflows/proto-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,12 @@ jobs:

- name: Setup build depends
run: |
go get github.com/grpc-ecosystem/grpc-gateway/v2/internal/[email protected]
go get google.golang.org/grpc/cmd/protoc-gen-go-grpc
go install github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway
go install github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2
go install google.golang.org/protobuf/cmd/protoc-gen-go
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc

- uses: bufbuild/[email protected]
go install github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway@latest
go install github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2@latest
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

- uses: bufbuild/[email protected]

- name: buf generate
working-directory: ./gctrpc
Expand Down
2 changes: 1 addition & 1 deletion backtester/btrpc/btrpc.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

198 changes: 90 additions & 108 deletions backtester/btrpc/btrpc.pb.gw.go

Large diffs are not rendered by default.

50 changes: 31 additions & 19 deletions backtester/btrpc/btrpc_grpc.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

94 changes: 39 additions & 55 deletions common/timedmutex/timed_mutex.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,77 +2,61 @@ package timedmutex

import (
"sync"
"sync/atomic"
"time"
)

// NewTimedMutex creates a new timed mutex with a
// specified duration
// TimedMutex is a blocking mutex which will unlock after a specified time
type TimedMutex struct {
// primary mutex is the main lock that will be unlocked after the duration
primary sync.Mutex
// secondary mutex is used to protect the timer
secondary sync.Mutex
timer *time.Timer
// primed is used to determine if the timer has been started this is
// slightly more performant than checking the timer directly and interacting
// with a RW mutex.
primed atomic.Bool
duration time.Duration
}

// NewTimedMutex creates a new timed mutex with a specified duration
func NewTimedMutex(length time.Duration) *TimedMutex {
return &TimedMutex{
duration: length,
}
return &TimedMutex{duration: length}
}

// LockForDuration will start a timer, lock the mutex,
// then allow the caller to continue
// LockForDuration will start a timer, lock the mutex, then allow the caller to continue
// After the duration, the mutex will be unlocked
func (t *TimedMutex) LockForDuration() {
var wg sync.WaitGroup
wg.Add(1)
go t.lockAndSetTimer(&wg)
wg.Wait()
}

func (t *TimedMutex) lockAndSetTimer(wg *sync.WaitGroup) {
t.mtx.Lock()
t.setTimer()
wg.Done()
t.primary.Lock()
if !t.primed.Swap(true) {
t.secondary.Lock()
t.timer = time.AfterFunc(t.duration, func() { t.primary.Unlock() })
t.secondary.Unlock()
} else {
// Timer C channel is not used with AfterFunc, so no need to drain.
t.secondary.Lock()
t.timer.Reset(t.duration)
t.secondary.Unlock()
}
}

// UnlockIfLocked will unlock the mutex if its currently locked
// Will return true if successfully unlocked
// UnlockIfLocked will unlock the mutex if its currently locked Will return true
// if successfully unlocked
func (t *TimedMutex) UnlockIfLocked() bool {
if t.isTimerNil() {
if !t.primed.Load() {
return false
}

if !t.stopTimer() {
return false
}
t.mtx.Unlock()
return true
}
t.secondary.Lock()
wasStoppedByCall := t.timer.Stop()
t.secondary.Unlock()

// stopTimer will return true if timer has been stopped by this command
// If the timer has expired, clear the channel
func (t *TimedMutex) stopTimer() bool {
t.timerLock.Lock()
defer t.timerLock.Unlock()
if !t.timer.Stop() {
select {
case <-t.timer.C:
default:
}
if !wasStoppedByCall {
// Timer has already fired and the mutex has been unlocked.
// Timer C channel is not used with AfterFunc, so no need to drain.
return false
}
t.primary.Unlock()
return true
}

// isTimerNil safely read locks to detect nil
func (t *TimedMutex) isTimerNil() bool {
t.timerLock.RLock()
isNil := t.timer == nil
t.timerLock.RUnlock()
return isNil
}

// setTimer safely locks and sets a timer
// which will automatically execute a mutex unlock
// once timer expires
func (t *TimedMutex) setTimer() {
t.timerLock.Lock()
t.timer = time.AfterFunc(t.duration, func() {
t.mtx.Unlock()
})
t.timerLock.Unlock()
}
33 changes: 32 additions & 1 deletion common/timedmutex/timed_mutex_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,41 @@ import (
"time"
)

// 1000000 1074 ns/op 136 B/op 4 allocs/op (prev)
// 2423571 503.9 ns/op 0 B/op 0 allocs/op (current)
func BenchmarkTimedMutexTime(b *testing.B) {
tm := NewTimedMutex(20 * time.Millisecond)
tm := NewTimedMutex(0)
for i := 0; i < b.N; i++ {
tm.LockForDuration()
}
}

// 352309195 3.194 ns/op 0 B/op 0 allocs/op (prev)
// 927051118 1.298 ns/op 0 B/op 0 allocs/op
func BenchmarkTimedMutexTimeUnlockNotPrimed(b *testing.B) {
tm := NewTimedMutex(0)
for i := 0; i < b.N; i++ {
tm.UnlockIfLocked()
}
}

// 95322825 15.51 ns/op 0 B/op 0 allocs/op (prev)
// 239158972 4.621 ns/op 0 B/op 0 allocs/op
func BenchmarkTimedMutexTimeUnlockPrimed(b *testing.B) {
tm := NewTimedMutex(0)
tm.LockForDuration()
for i := 0; i < b.N; i++ {
tm.UnlockIfLocked()
}
}

// 1000000 1193 ns/op 136 B/op 4 allocs/op (prev)
// 38592405 36.12 ns/op 0 B/op 0 allocs/op
func BenchmarkTimedMutexTimeLinearInteraction(b *testing.B) {
tm := NewTimedMutex(0)
for i := 0; i < b.N; i++ {
tm.LockForDuration()
tm.UnlockIfLocked()
}
}

Expand Down
15 changes: 0 additions & 15 deletions common/timedmutex/timed_mutex_types.go

This file was deleted.

Loading