|
| 1 | +# What is it ? |
| 2 | + |
| 3 | +`pg-node` is an **experimental** in-memory emulation of a postgres database. |
| 4 | + |
| 5 | +It works both in node or in browser. |
| 6 | + |
| 7 | +## See it in action with [pg-mem playground](https://oguimbal.github.io/pg-mem-playground/) |
| 8 | + |
| 9 | +## DISCLAIMER |
| 10 | + |
| 11 | +The syntax parser is home-made. Which means that some features are not implemented, and will be considered as invalid syntaxes. |
| 12 | + |
| 13 | +This lib is quite new, so forgive it if some obivious pg syntax is not supported ! |
| 14 | + |
| 15 | +... And open an issue if you feel like a feature should be implemented :) |
| 16 | + |
| 17 | +Moreover, even if I wrote hundreds of tests, keep in mind that this implementation is a best effort to replicate PG. |
| 18 | +Keep an eye on your query results if you perform complex queries. |
| 19 | +Please file issues if some results seem incoherent with what should be returned. |
| 20 | + |
| 21 | +Finally, I invite you to read the below section to have an idea of you can or cannot do. |
| 22 | + |
| 23 | + |
| 24 | +# Supported features |
| 25 | + |
| 26 | +It supports: |
| 27 | +- [x] Transactions |
| 28 | +- [x] Indices, somewhat (on "simple" requests) |
| 29 | +- [x] Basic data types (json, dates, ...) |
| 30 | +- [x] Joins, group bys, ... |
| 31 | +- [x] Easy wrapper creator for [Typeorm](https://github.com/typeorm/typeorm), [pg-promise (pgp)](https://github.com/vitaly-t/pg-promise), [node-postgres (pg)](https://github.com/brianc/node-postgres), [pg-native](https://github.com/brianc/node-pg-native) |
| 32 | + |
| 33 | + |
| 34 | +It does not (yet) support (this is kind-of a todo list): |
| 35 | +- [ ] Gin Indices |
| 36 | +- [ ] Cartesian Joins |
| 37 | +- [ ] Most of the pg functions are not implemented - ask for them, [they're easy to implement](src/functions.ts) ! |
| 38 | +- [ ] Some [aggregations](src/transforms/aggregation.ts) are to be implemented (avg, count, ...) - easy job, but not yet done. |
| 39 | +- [ ] Stored procedures |
| 40 | +- [ ] Lots of small and not so small things (collate, timezones, tsqueries, custom types ...) |
| 41 | +- [ ] Introspection schema (it is faked - i.e. some table exist, but are empty - so Typeorm can inspect an introspect an empty db & create tables) |
| 42 | + |
| 43 | +... PR are open :) |
| 44 | + |
| 45 | +# Usage |
| 46 | + |
| 47 | + |
| 48 | +## Using NodeJS |
| 49 | +As always, it stats with an: |
| 50 | + |
| 51 | +```bash |
| 52 | +npm i pg-mem --save |
| 53 | +``` |
| 54 | + |
| 55 | +Then, assuming you're using something like Webpack if you're targetting a browser: |
| 56 | + |
| 57 | +```typescript |
| 58 | +import { newDb } from 'pg-mem'; |
| 59 | + |
| 60 | +const db = newDb(); |
| 61 | +db.public.many(/* put some sql here */) |
| 62 | +``` |
| 63 | + |
| 64 | +## Using Deno |
| 65 | + |
| 66 | +Pretty straightforward :) |
| 67 | + |
| 68 | +```typescript |
| 69 | +import { newDb } from 'https://deno.land/x/pg-mem/mod.ts'; |
| 70 | + |
| 71 | +const db = newDb(); |
| 72 | +db.public.many(/* put some sql here */) |
| 73 | +``` |
| 74 | + |
| 75 | +# Features |
| 76 | + |
| 77 | +## Rollback to a previous state |
| 78 | + |
| 79 | +`pg-mem` uses immutable data structures ([here](https://www.npmjs.com/package/immutable) and [here](https://www.npmjs.com/package/functional-red-black-tree)), |
| 80 | +which means that you can have restore points for free ! |
| 81 | + |
| 82 | +This is super useful if you indend to use `pg-mem` to mock your database for unit tests. |
| 83 | +You could: |
| 84 | + |
| 85 | +1) Create your schema only once (which could be an heavy operation for a single unit test) |
| 86 | +2) Insert test data which will be shared by all test |
| 87 | +2) Create a restore point |
| 88 | +3) Run your tests with the same db instance, executing a `backup.restore()` before each test (which instantly resets db to the state it has after creating the restore point) |
| 89 | + |
| 90 | +Usage: |
| 91 | +```typescript |
| 92 | +const db = newDb(); |
| 93 | +db.public.none(`create table test(id text); |
| 94 | + insert into test values ('value');`); |
| 95 | +// create a restore point & mess with data |
| 96 | +const backup = db.backup(); |
| 97 | +db.public.none(`update test set id='new value';`) |
| 98 | +// restore it ! |
| 99 | +backup.restore(); |
| 100 | +db.public.many(`select * from test`) // => {test: 'value'} |
| 101 | +``` |
| 102 | + |
| 103 | +## pg-native |
| 104 | + |
| 105 | +You can ask `pg-mem` to get you an object wich implements the same behaviour as [pg-native](https://github.com/brianc/node-pg-native). |
| 106 | + |
| 107 | + |
| 108 | +```typescript |
| 109 | +// instead of |
| 110 | +import Client from 'pg-native'; |
| 111 | + |
| 112 | +// use: |
| 113 | +import {newDb} from 'pg-mem'; |
| 114 | +const Client = newDb.adapters.createPgNative(); |
| 115 | +``` |
| 116 | + |
| 117 | + |
| 118 | +## node-postgres (pg) |
| 119 | + |
| 120 | +You can use `pg-mem` to get a memory version of the [node-postgres (pg)](https://github.com/brianc/node-postgres) module. |
| 121 | + |
| 122 | +```typescript |
| 123 | +// instead of |
| 124 | +import {Client} from 'pg'; |
| 125 | + |
| 126 | +// use: |
| 127 | +import {newDb} from 'pg-mem'; |
| 128 | +const {Client} = newDb.adapters.createPg(); |
| 129 | +``` |
| 130 | + |
| 131 | + |
| 132 | +## pg-promise (pgp) |
| 133 | + |
| 134 | +You can ask `pg-mem` to get you a [pg-promise](https://github.com/vitaly-t/pg-promise) instance bound to this db. |
| 135 | + |
| 136 | +Given that pg-promise [does not provide](https://github.com/vitaly-t/pg-promise/issues/743) any way to be hooked, [I had to fork it](https://github.com/oguimbal/pg-promise). |
| 137 | +You must install this fork in order to use this (not necessarily use it in production): |
| 138 | + |
| 139 | +```bash |
| 140 | +npm i @oguimbal/pg-promise -D |
| 141 | +``` |
| 142 | + |
| 143 | +Then: |
| 144 | + |
| 145 | +```typescript |
| 146 | +// instead of |
| 147 | +import pgp from 'pg-promise'; |
| 148 | +const pg = pgp(opts) |
| 149 | + |
| 150 | +// use: |
| 151 | +import {newDb} from 'pg-mem'; |
| 152 | +const pg = await newDb.adapters.createPgPromise(); |
| 153 | + |
| 154 | +// then use it like you would with pg-promise |
| 155 | +await pg.connect(); |
| 156 | +``` |
| 157 | + |
| 158 | + |
| 159 | +## slonik |
| 160 | + |
| 161 | +You can use `pg-mem` to get a memory version of a [slonik](https://github.com/gajus/slonik) pool. |
| 162 | + |
| 163 | +```typescript |
| 164 | +// instead of |
| 165 | +import {createPool} from 'slonik'; |
| 166 | +const pool = createPool(/* args */); |
| 167 | + |
| 168 | +// use: |
| 169 | +import {newDb} from 'pg-mem'; |
| 170 | +const pool = newDb.adapters.createSlonik(); |
| 171 | +``` |
| 172 | + |
| 173 | + |
| 174 | +## Typeorm |
| 175 | + |
| 176 | +You can use `pg-mem` as a backend database for [Typeorm](https://github.com/typeorm/typeorm), [node-postgres (pg)](https://github.com/brianc/node-postgres). |
| 177 | + |
| 178 | +Usage: |
| 179 | +```typescript |
| 180 | +const db = newDb(); |
| 181 | +const connection = await db.adapters.createTypeormConnection({ |
| 182 | + type: 'postgres', |
| 183 | + entities: [/* your entities here ! */] |
| 184 | +}) |
| 185 | + |
| 186 | +// create schema |
| 187 | +await connection.synchronize(); |
| 188 | + |
| 189 | +// => you now can user your typeorm connection ! |
| 190 | +``` |
| 191 | + |
| 192 | +See detailed examples [here](samples/typeorm/simple.ts) and [here](samples/typeorm/joins.ts). |
| 193 | + |
| 194 | +See restore points (section above) to avoid running schema creation (`.synchronize()`) on each test. |
| 195 | + |
| 196 | +__NB: Restore points only work if the schema has not been changed after the restore point has been created__ |
| 197 | + |
| 198 | +note: You must install `typeorm` module first. |
| 199 | + |
| 200 | +# Inspection |
| 201 | + |
| 202 | +## Subscriptions |
| 203 | +You can subscribe to some events, like: |
| 204 | + |
| 205 | +```typescript |
| 206 | +const db = newDb(); |
| 207 | + |
| 208 | +// called on each successful sql request |
| 209 | +db.on('query', sql => { }); |
| 210 | +// called on each failed sql request |
| 211 | +db.on('query-failed', sql => { }); |
| 212 | +// called on schema changes |
| 213 | +db.on('schema-change', () => {}); |
| 214 | +``` |
| 215 | + |
| 216 | +## Experimental subscriptions |
| 217 | + |
| 218 | +`pg-mem` implements a basic support for indices. |
| 219 | + |
| 220 | +These handlers are called when a request cannot be optimized using one of the created indices. |
| 221 | + |
| 222 | +However, a real postgres instance will be much smarter to optimize its requests... so when `pg-mem` says "this request does not use an index", dont take my word for it. |
| 223 | + |
| 224 | +```typescript |
| 225 | +// called when a table is iterated entierly (ex: 'select * from data where notIndex=3' triggers it) |
| 226 | +db.on('seq-scan', () => {}); |
| 227 | + |
| 228 | +// same, but on a specific table |
| 229 | +db.getTable('myTable').on('seq-scan', () = {}); |
| 230 | + |
| 231 | +// will be called if pg-mem did not find any way to optimize a join |
| 232 | +db.on('catastrophic-join-optimization', () => {}); |
| 233 | +``` |
| 234 | + |
| 235 | +# Development |
| 236 | + |
| 237 | +Pull requests are welcome :) |
| 238 | + |
| 239 | +To start hacking this lib, you'll have to: |
| 240 | +- Use vscode |
| 241 | +- Install [mocha test explorer with HMR support](https://marketplace.visualstudio.com/items?itemName=oguimbal.vscode-mocha-test-adapter) extension |
| 242 | +- `npm start` |
| 243 | +- Reload unit tests in vscode |
| 244 | + |
| 245 | +... once done, tests should appear. HMR is on, which means that changes in your code are instantly propagated to unit tests. |
| 246 | +This allows for ultra fast development cycles (running tests takes less than 1 sec). |
| 247 | + |
| 248 | +To debug tests: Just hit "run" (F5, or whatever)... vscode should attach the mocha worker. Then run the test you want to debug. |
0 commit comments