-
Notifications
You must be signed in to change notification settings - Fork 233
/
Copy pathGEN_README.md.tmpl
347 lines (283 loc) · 15.5 KB
/
GEN_README.md.tmpl
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
[comment]: <> (This is a generated file please edit source in ./templates)
[comment]: <> (All modification will be lost, you have been warned)
[comment]: <> ()
## gen
[](https://opensource.org/licenses/Apache-3.0) [](http://godoc.org/github.com/smallnest/gen) [](https://travis-ci.org/smallnest/gen) [](https://goreportcard.com/report/github.com/smallnest/gen)
The gen tool produces a CRUD (Create, read, update and delete) REST api project template from a given database. The gen tool will
connect to the db connection string analyze the database and generate the code based on the flags provided.
By reading details from the database about the column structure, gen generates a go compatible struct type
with the required column names, data types, and annotations.
It supports [gorm](https://github.com/jinzhu/gorm) tags and implements some usable methods. Generated data types include support for nullable columns [sql.NullX types](https://golang.org/pkg/database/sql/#NullBool) or [guregu null.X types](https://github.com/guregu/null)
and the expected basic built in go types.
`gen` is based / inspired by the work of Seth Shelnutt's [db2struct](https://github.com/Shelnutt2/db2struct), and Db2Struct is based/inspired by the work of ChimeraCoder's gojson package [gojson](https://github.com/ChimeraCoder/gojson).
## CRUD Generation
This is a sample table contained within the ./example/sample.db Sqlite3 database. Using `gen` will generate the following struct.
```sql
CREATE TABLE "albums"
(
[AlbumId] INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
[Title] NVARCHAR(160) NOT NULL,
[ArtistId] INTEGER NOT NULL,
FOREIGN KEY ([ArtistId]) REFERENCES "artists" ([ArtistId])
ON DELETE NO ACTION ON UPDATE NO ACTION
)
```
#### Transforms into
```go
type Album struct {
//[ 0] AlbumId integer null: false primary: true auto: true col: integer len: -1 default: []
AlbumID int `gorm:"primary_key;AUTO_INCREMENT;column:AlbumId;type:INTEGER;" json:"album_id" db:"AlbumId" protobuf:"int32,0,opt,name=album_id"`
//[ 1] Title nvarchar(160) null: false primary: false auto: false col: nvarchar len: 160 default: []
Title string `gorm:"column:Title;type:NVARCHAR(160);size:160;" json:"title" db:"Title" protobuf:"string,1,opt,name=title"`
//[ 2] ArtistId integer null: false primary: false auto: false col: integer len: -1 default: []
ArtistID int `gorm:"column:ArtistId;type:INTEGER;" json:"artist_id" db:"ArtistId" protobuf:"int32,2,opt,name=artist_id"`
}
```
Code generation for a complete CRUD rest project is possible with DAO crud functions, http handlers, makefile, sample server are available. Check out some of the [code generated samples](#Generated-Samples).
## Binary Installation
```BASH
## install gen tool (should be installed to ~/go/bin, make sure ~/go/bin is in your path.
$ go get -u github.com/smallnest/gen
## download sample sqlite database
$ wget https://github.com/smallnest/gen/raw/master/example/sample.db
## generate code based on the sqlite database (project will be contained within the ./example dir)
$ gen --sqltype=sqlite3 \
--connstr "./sample.db" \
--database main \
--json \
--gorm \
--guregu \
--rest \
--out ./example \
--module example.com/rest/example \
--mod \
--server \
--makefile \
--json-fmt=snake \
--generate-dao \
--generate-proj \
--overwrite
## build example code (build process will install packr2 if not installed)
$ cd ./example
$ make example
## binary will be located at ./bin/example
## when launching make sure that the SQLite file sample.db is located in the same dir as the binary
$ cp ../../sample.db .
$ ./example
## Open a browser to {{$.serverScheme}}://{{$.serverHost}}{{if ne $.serverPort 80}}:{{$.serverPort}}{{end}}/swagger/index.html
## Use wget/curl/httpie to fetch via command line
http http://localhost:8080/albums
curl http://localhost:8080/artists
```
## Usage
```console
{{.GenHelp}}
```
## Building
The project contains a makefile for easy building and common tasks.
* `make help` - list available targets
* `make build` - generate the binary `./gen`
* `make example` - run the gen process on the example SqlLite db located in ./examples place the sources in ./example
Other targets exist for dev tasks.
## Example
The project provides a sample SQLite database in the `./example` directory. From the project `Makefile` can be used to generate the example code.
```.bash
make example
```
The generated project will contain the following code under the `./example` directory.
* Makefile
* useful Makefile for installing tools building project etc. Issue `make` to display help output.
* .gitignore
* git ignore for go project
* go.mod
* go module setup, pass `--module` flag for setting the project module default `example.com/example`
* README.md
* Project readme
* app/server/main.go
* Sample Gin Server, with swagger init and comments
* api/<table name>.go
* REST crud controllers
* dao/<table name>.go
* DAO functions providing CRUD access to database
* model/<table name>.go
* Structs representing a row for each database table
#### Generated Samples
* [GORM DAO CRUD Functions](./code_dao_gorm.md)
* [SQLX DAO CRUD Functions](./code_dao_sqlx.md)
* [Http CRUD Handlers](./code_http.md)
* [Model](./code_model.md)
* [Protobuf Definition](./code_protobuf.md)
The REST api server utilizes the Gin framework, GORM db api and Swag for providing swagger documentation
* [Gin](https://github.com/gin-gonic/gin)
* [Swaggo](https://github.com/swaggo/swag)
* [Gorm](https://github.com/jinzhu/gorm)
* [packr2](https://github.com/gobuffalo/packr)
## Supported Databases
Currently Supported,
- MariaDB
- MySQL
- PostgreSQL
- Microsoft SQL Server
- SQLite
Planned Support
- Oracle
## Supported Data Types
Most data types are supported, for Mysql, Postgres, SQLite and MS SQL. `gen` uses a mapping json file that can be used to add mapping types. By default, the internal mapping file is loaded and processed. If can be overwritten or additional types added by using the `--mapping=extra.json` command line option.
The default `mapping.json` file is located within the ./templates dir. Use `gen --save=./templates` to save the contents of the templates to `./templates`.
Below is a portion of the mapping file, showing the mapping for `varchar`.
```json
{
"sql_type": "varchar",
"go_type": "string",
"protobuf_type": "bytes",
"guregu_type": "null.String",
"go_nullable_type": "sql.NullString"
}
```
## Advanced
The `gen` tool provides functionality to layout your own project format. Users have 2 options.
* Provide local templates with the `--templateDir=` option - this will generate code using the local templates. Templates can either be exported from `gen`
via the command `gen --save ./mytemplates`. This will save the embedded templates for local editing. Then you would specify the `--templateDir=` option when generating a project.
* Passing `--exec=../sample.gen` on the command line will load the `sample.gen` script and execute it. The script has access to the table information and other info passed to `gen`. This allows developers to customize the generation of code. You could loop through the list of tables and invoke
`GenerateTableFile` or `GenerateFile`.
You can also populate the context used by templates with extra data by passing the `--contect=<json file>` option. The json file will be used to populate the context used when parsing templates.
```gotemplate
// Loop through tables and print out table name and various forms of the table name
{{.AdvancesSample}}
// GenerateTableFile(tableName, templateFilename, outputDirectory, outputFileName string, formatOutput bool)
// GenerateFile(templateFilename, outputDirectory, outputFileName string, formatOutput bool) string
The following info is available within use of the exec template.
{{ range $key, $value := . }}
{{ printf "%#-25v" $key }} {{ printf "%-30T %#v" $value $value }}{{ end }}
```
## Struct naming
The ability exists to set a template that will be used for generating a struct name. By passing the flag `--model_naming={{"{{.}}"}}`
The struct will be named the table name. Various functions can be used in the template to modify the name such as
You can use the argument `--name_test=user` in conjunction with the `--model_naming` or `--file_name`, to view what the naming would be.
| Function | Table Name | Output
|---|---|---|
|singular |Users | `{{singular "Users" }}` |
|pluralize |Users | `{{pluralize "Users" }}` |
|title |Users | `{{title "Users" }}` |
|toLower |Users | `{{toLower "Users" }}` |
|toUpper |Users | `{{toUpper "Users" }}` |
|toLowerCamelCase |Users | `{{toLowerCamelCase "Users" }}` |
|toUpperCamelCase |Users | `{{toUpperCamelCase "Users" }}` |
|toSnakeCase |Users | `{{toSnakeCase "Users" }}` |
### Struct naming Examples
Table Name: registration_source
| Model Naming Format | Generated Struct Name
|---|---|
|`{{"{{"}}.{{"}}"}}` | {{"registration_source"}} |
|`Struct{{"{{"}}.{{"}}"}}` | Struct{{"registration_source"}} |
|`Struct{{"{{ singular "}}.{{"}}"}}` | Struct{{ singular "registration_source"}} |
|`Struct{{"{{ toLowerCamelCase "}}.{{"}}"}}` | Struct{{ toLowerCamelCase "registration_source"}} |
|`Struct{{"{{ toUpperCamelCase "}}.{{"}}"}}` | Struct{{ toUpperCamelCase "registration_source"}} |
|`Struct{{"{{ toSnakeCase "}}.{{"}}"}}` | Struct{{ toSnakeCase "registration_source"}} |
|`Struct{{"{{ toLowerCamelCase "}}.{{"}}"}}` | Struct{{ toLowerCamelCase "table_registration_source"}} |
|`Struct{{"{{ toUpperCamelCase "}}.{{"}}"}}` | Struct{{ toUpperCamelCase "table_registration_source"}} |
|`Struct{{"{{ toSnakeCase "}}.{{"}}"}}` | Struct{{ toSnakeCase "table_registration_source"}} |
|`Struct{{"{{ toSnakeCase ( replace . \"table_\" \"\") }}`"}} | Struct{{ toSnakeCase ( replace "table_registration_source" "table_" "") }} |
## Notes
- MySql, Mssql, Postgres and Sqlite have a database metadata fetcher that will query the db, and update the auto increment, primary key and nullable info for the gorm annotation.
- Tables that have a non-standard primary key (NON integer based or String) the table will be ignored.
## DB Meta Data Loading
| DB | Type | Nullable | Primary Key | Auto Increment | Column Len | default Value| create ddl
|---|---|---|---|---|---|---|---|
|sqlite |y | y | y | y | y | y| y
|postgres |y | y | y | y | y | y| n
|mysql |y | y | y | y | y | y| y
|ms sql |y | y | y | y | y | y| n
## Version History
- v0.9.25 (07/26/2020)
- Adhere json-fmt flag for all JSON response so when camel or lower_camel is specified, fields name in GetAll variant and DDL info will also have the same name format
- Fix: Build information embedded through linker in Makefile is not consistent with the variable defined in main file.
- Added --scheme and --listen options. This allows compiled binary to be used behind reverse proxy.
- In addition, template for generating URL was fixed, i.e. when PORT is 80, then PORT is omitted from URL segment.
- v0.9.24 (07/13/2020)
- Fixed array bounds issue parsing mysql db meta
- v0.9.23 (07/10/2020)
- Added postgres types: bigserial, serial, smallserial, bigserial, float4 to mapping.json
- v0.9.22 (07/08/2020)
- Modified gogo.proto check to use GOPATH not hardcoded.
- Updated gen to error exit on first error encountered
- Added color output for error
- Added --no-color option for non colorized output
- v0.9.21 (07/07/2020)
- Repacking templates, update version number in info.
- v0.9.20 (07/07/2020)
- Fixed render error in router.go.tmpl
- upgraded project to use go.mod 1.14
- v0.9.19 (07/07/2020)
- Added --windows flag to write files with CRLF windows line endings, otherwise they are all unix based LF line endings
- v0.9.18 (06/30/2020)
- Fixed naming in templates away from hard coded model package.
- v0.9.17 (06/30/2020)
- Refactored template loading, to better report error in template
- Added option to run gofmt on output directory
- v0.9.16 (06/29/2020)
- Fixes to router.go.tmpl from calvinchengx
- Added postgres db support for inet and timestamptz
- v0.9.15 (06/23/2020)
- Code cleanup using gofmt name suggestions.
- Template updates for generated code cleanup using gofmt name suggestions.
- v0.9.14 (06/23/2020)
- Added model comment on field line if available from database.
- Added exposing TableInfo via api call.
- v0.9.13 (06/22/2020)
- fixed closing of connections via defer
- bug fixes in sqlx generated code
- v0.9.12 (06/14/2020)
- SQLX changed MustExec to Exec and checking/returning error
- Updated field renaming if duplicated, need more elegant renaming solution.
- Added exclude to test.sh
- v0.9.11 (06/13/2020)
- Added ability to pass field, model and file naming format
- updated test scripts
- Fixed sqlx sql query placeholders
- v0.9.10 (06/11/2020)
- Bug fix with retrieving varchar length from mysql
- Added support for mysql unsigned decimal - maps to float
- v0.9.9 (06/11/2020)
- Fixed issue with mysql and table named `order`
- Fixed internals in GetAll generation in gorm and sqlx.
- v0.9.8 (06/10/2020)
- Added ability to set file naming convention for models, dao, apis and grpc `--file_naming={{"{{.}}"}}`
- Added ability to set struct naming convention `--model_naming={{"{{.}}"}}`
- Fixed bug with Makefile generation removing quoted conn string in `make regen`
- v0.9.7 (06/09/2020)
- Added grpc server generation - WIP (looking for code improvements)
- Added ability to exclude tables
- Added support for unsigned from mysql ddl.
- v0.9.6 (06/08/2020)
- Updated SQLX codegen
- Updated templates to split code gen functions into seperate files
- Added code_dao_gorm, code_dao_sqlx to be generated from templates
- v0.9.5 (05/16/2020)
- Added SQLX codegen by default, split dao templates.
- Renamed templates
- v0.9.4 (05/15/2020)
- Documentation updates, samples etc.
- v0.9.3 (05/14/2020)
- Template bug fixes, when using custom api, dao and model package.
- Set primary key if not set to the first column
- Skip code gen if primary key column is not int or string
- validated codegen for mysql, mssql, postgres and sqlite3
- Fixed file naming if table ends with _test.go renames to _tst.go
- Fix for duplicate field names in struct due to renaming
- Added Notes for columns and tables for situations where a primary key is set since not defined in db
- Fixed issue when model contained field that had were named the same as funcs within model.
- v0.9.2 (05/12/2020)
- Code cleanup gofmt, etc.
- v0.9.1 (05/12/2020)
- v0.9 (05/12/2020)
- updated db meta data loading fetching default values
- added default value to GORM tags
- Added protobuf .proto generation
- Added test app to display meta data
- Cleanup DDL generation
- Added support for varchar2, datetime2, float8, USER_DEFINED
- v0.5
## Contributors
- [alexj212](https://github.com/alexj212) - a big thanks to alexj212 for his contributions
See more contributors: [contributors](https://github.com/smallnest/gen/graphs/contributors)