Skip to content

Commit 183f891

Browse files
rajeevsraorajeevsrao
committed
Update onnx-tensorrt and copyright headers (2021)
Signed-off-by: Rajeev Rao <[email protected]>
1 parent da988fd commit 183f891

File tree

470 files changed

+765
-763
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

470 files changed

+765
-763
lines changed

‎.gitmodules

+1-1
Original file line numberDiff line numberDiff line change
@@ -9,4 +9,4 @@
99
[submodule "parsers/onnx"]
1010
path = parsers/onnx
1111
url = https://github.com/onnx/onnx-tensorrt.git
12-
branch = 7.2.1
12+
branch = master

‎CMakeLists.txt

+1-2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#
2-
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
2+
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
33
#
44
# Licensed under the Apache License, Version 2.0 (the "License");
55
# you may not use this file except in compliance with the License.
@@ -194,4 +194,3 @@ endif()
194194
if(BUILD_SAMPLES)
195195
add_subdirectory(samples)
196196
endif()
197-

‎CODING-GUIDELINES.md

+10-10
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ if (nbInputs == kNbInputsWBias) {/*...*/}
7676
* Public member variables do not require the 'm' prefix but it is highly encouraged to use the prefix when needed to improve code clarity, especially in cases where the class is a base class in an inheritance chain.
7777

7878
8. Constants
79-
* Enumerations, global constants, static constants at class-scope and function-scope magic-number/literal constants are uppercase snakecase with prefix 'k':
79+
* Enumerations, global constants, static constants at class-scope and function-scope magic-number/literal constants are uppercase snakecase with prefix 'k':
8080
```cpp
8181
const int kDIGIT_NUM = 10;
8282
```
@@ -103,7 +103,7 @@ Notes:
103103

104104
#### Formatting
105105
1. Use the [LLVM clang-format](https://clang.llvm.org/docs/ClangFormat.html) tool for formatting your changes prior to submitting the PR.
106-
2. Use a maximum of 120 characters per line. The auto formatting tool will wrap longer lines.
106+
2. Use a maximum of 120 characters per line. The auto formatting tool will wrap longer lines.
107107
3. Exceptions to formatting violations must be justified on a per-case basis. Bypassing the formatting rules is discouraged, but can be achieved for exceptions as follows:
108108
```cpp
109109
// clang-format off
@@ -133,7 +133,7 @@ doSomeOperation(/* checkForErrors = */ false);
133133
```cpp
134134
//! This is a Doxygen comment
135135
//! in C++ style
136-
136+
137137
struct Foo
138138
{
139139
int x; //!< This is a Doxygen comment for members
@@ -149,7 +149,7 @@ struct Foo
149149
#endif
150150
```
151151

152-
```cpp
152+
```cpp
153153
// Alternative: use a macro which evaluates to a noop in release code.
154154
#if DEBUG_CONVOLUTION_INSTRUMENTATION
155155
# define DEBUG_CONV_CODE(x) x
@@ -235,7 +235,7 @@ switch (x) case 4: if (y) case 5: return 0; else default: return 1;
235235
switch (x)
236236
{
237237
case 0: // Fall-through allowed from case 0: to case 1: since case 0 is empty.
238-
case 1:
238+
case 1:
239239
a();
240240
b();
241241
break;
@@ -250,7 +250,7 @@ case 5:
250250
c();
251251
throw 42; // Terminating with throw is okay
252252
default:
253-
throw 42;
253+
throw 42;
254254
}
255255
```
256256

@@ -297,7 +297,7 @@ case 1:
297297
#### Preprocessor Directives
298298
1. *MISRA C++: 2008 Rule 16-0-2*
299299
`#define` and `#undef` of macros should be done only at global namespace.
300-
2. Avoid the use of `#ifdef` and `#ifndef` directives (except in the case of header include guards). Prefer to use `#if defined(...)` or `#if !defined(...)` instead. The latter syntax is more consistent with C syntax, and allows you to use more complicated preprocessor conditionals, e.g.:
300+
2. Avoid the use of `#ifdef` and `#ifndef` directives (except in the case of header include guards). Prefer to use `#if defined(...)` or `#if !defined(...)` instead. The latter syntax is more consistent with C syntax, and allows you to use more complicated preprocessor conditionals, e.g.:
301301
```cpp
302302
#if defined(FOO) || defined(BAR)
303303
void foo();
@@ -343,14 +343,14 @@ for (size_t i = 0; i < mTensors.size(); ++i) // preferred style
343343
```
344344
* Using only signed integers for the above would lead to prolixity and perhaps unsafe narrowing:
345345
```cpp
346-
for (int i = 0; i < static_cast<int>(mTensors.size()); ++i)
346+
for (int i = 0; i < static_cast<int>(mTensors.size()); ++i)
347347
```
348348

349349

350350
#### Special Considerations for API
351351
1. The API consists, with very few exceptions, of methodless structs and pure virtual interface classes.
352352
2. API class methods should be either virtual or inline.
353-
3. The API does not use integral types with platform-dependent sizes, other than `int`, `unsigned`, and `bool`. `size_t` should be used only for sizes of memory buffers.
353+
3. The API does not use integral types with platform-dependent sizes, other than `int`, `unsigned`, and `bool`. `size_t` should be used only for sizes of memory buffers.
354354
4. The API does not use any aggregate types (e.g. `std::string`) which may be compiled differently with different compilers and libraries.
355355
5. The API minimizes dependencies on system headers - currently only `<cstddef>` and `<cstdint>`.
356356
6. Memory ownership may not be transferred across API boundaries - any memory allocated inside a library must be freed inside the library.
@@ -419,7 +419,7 @@ char const * const errStr = getErrorStr(status);
419419
1. All TensorRT Open Source Software code should contain an NVIDIA copyright header that includes the current year. The following block of text should be prepended to the top of all OSS files. This includes .cpp, .h, .cu, .py, and any other source files which are compiled or interpreted.
420420
```cpp
421421
/*
422-
* Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
422+
* Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
423423
*
424424
* Licensed under the Apache License, Version 2.0 (the "License");
425425
* you may not use this file except in compliance with the License.

‎cmake/modules/find_library_create_target.cmake

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#
2-
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
2+
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
33
#
44
# Licensed under the Apache License, Version 2.0 (the "License");
55
# you may not use this file except in compliance with the License.

‎cmake/modules/set_ifndef.cmake

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#
2-
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
2+
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
33
#
44
# Licensed under the Apache License, Version 2.0 (the "License");
55
# you may not use this file except in compliance with the License.

‎cmake/toolchains/cmake_aarch64-android.toolchain

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#
2-
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
2+
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
33
#
44
# Licensed under the Apache License, Version 2.0 (the "License");
55
# you may not use this file except in compliance with the License.
@@ -40,7 +40,7 @@ set(CMAKE_CUDA_COMPILER_FORCED TRUE)
4040

4141
set(CUDA_LIBS -L${CUDA_ROOT}/lib64)
4242

43-
set(ADDITIONAL_PLATFORM_LIB_FLAGS ${CUDA_LIBS} -lcublas -lcudart -lnvToolsExt -lculibos -lcudadevrt -llog)
43+
set(ADDITIONAL_PLATFORM_LIB_FLAGS ${CUDA_LIBS} -lcublas -lcudart -lnvToolsExt -lculibos -lcudadevrt -llog)
4444

4545

4646
set(DISABLE_SWIG TRUE)

‎cmake/toolchains/cmake_aarch64.toolchain

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#
2-
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
2+
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
33
#
44
# Licensed under the Apache License, Version 2.0 (the "License");
55
# you may not use this file except in compliance with the License.

‎cmake/toolchains/cmake_ppc64le.toolchain

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#
2-
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
2+
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
33
#
44
# Licensed under the Apache License, Version 2.0 (the "License");
55
# you may not use this file except in compliance with the License.

‎cmake/toolchains/cmake_qnx.toolchain

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#
2-
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
2+
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
33
#
44
# Licensed under the Apache License, Version 2.0 (the "License");
55
# you may not use this file except in compliance with the License.

‎cmake/toolchains/cmake_x64_win.toolchain

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#
2-
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
2+
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
33
#
44
# Licensed under the Apache License, Version 2.0 (the "License");
55
# you may not use this file except in compliance with the License.
@@ -38,7 +38,7 @@ set(W10_LINKER ${MSVC_COMPILER_DIR}/bin/amd64/link)
3838

3939

4040
set(CMAKE_CUDA_HOST_COMPILER ${CMAKE_NVCC_COMPILER} CACHE STRING "" FORCE)
41-
41+
4242
set(ADDITIONAL_PLATFORM_INCL_FLAGS "-I${MSVC_COMPILER_DIR}/include -I${MSVC_COMPILER_DIR}/../ucrt/include")
4343
set(ADDITIONAL_PLATFORM_LIB_FLAGS ${ADDITIONAL_PLATFORM_LIB_FLAGS} "-LIBPATH:${NV_TOOLS}/ddk/wddmv2/official/17134/Lib/10.0.17134.0/um/x64")
4444
set(ADDITIONAL_PLATFORM_LIB_FLAGS ${ADDITIONAL_PLATFORM_LIB_FLAGS} "-LIBPATH:${MSVC_COMPILER_DIR}/lib/amd64" )

‎cmake/toolchains/cmake_x86_64.toolchain

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#
2-
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
2+
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
33
#
44
# Licensed under the Apache License, Version 2.0 (the "License");
55
# you may not use this file except in compliance with the License.

‎demo/BERT/CMakeLists.txt

+3-4
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#
2-
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
2+
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
33
#
44
# Licensed under the Apache License, Version 2.0 (the "License");
55
# you may not use this file except in compliance with the License.
@@ -49,7 +49,7 @@ link_directories(
4949
${TRT_LIB_DIR}
5050
)
5151

52-
pybind11_add_module(infer_c
52+
pybind11_add_module(infer_c
5353
infer_c/infer_c.cpp
5454
infer_c/logging.cpp
5555
)
@@ -64,9 +64,8 @@ add_executable(perf
6464
infer_c/logging.cpp
6565
)
6666

67-
target_link_libraries(perf
67+
target_link_libraries(perf
6868
${CUDA_LIBRARIES}
6969
nvinfer
7070
nvinfer_plugin
7171
)
72-

‎demo/BERT/builder.py

+7-7
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#!/usr/bin/env python3
22
#
3-
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
3+
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
44
#
55
# Licensed under the Apache License, Version 2.0 (the "License");
66
# you may not use this file except in compliance with the License.
@@ -229,7 +229,7 @@ def transformer_layer_opt(prefix, config, init_dict, network, input_tensor, imas
229229
hidden_size = idims[2]
230230

231231
if config.use_qat:
232-
dr_input = init_dict[prefix + 'attention_self_query_input_amax']
232+
dr_input = init_dict[prefix + 'attention_self_query_input_amax']
233233
assert(dr_input ==init_dict[prefix + 'attention_self_key_input_amax'] )
234234
assert(dr_input ==init_dict[prefix + 'attention_self_value_input_amax'] )
235235
input_tensor.set_dynamic_range(-dr_input, dr_input)
@@ -293,7 +293,7 @@ def transformer_layer_opt(prefix, config, init_dict, network, input_tensor, imas
293293
dr_gelu = init_dict[prefix + 'output_dense_input_amax']
294294
set_output_range(gelu_layer, dr_gelu)
295295
else:
296-
# use gelu10 according to whitepaper http://arxiv.org/abs/2004.09602
296+
# use gelu10 according to whitepaper http://arxiv.org/abs/2004.09602
297297
set_output_range(gelu_layer, 10)
298298

299299
# FC2
@@ -445,7 +445,7 @@ def onnx_to_trt_name(onnx_name):
445445
toks = [t.strip('_') for t in onnx_name.split('.')]
446446
if toks[0] == 'bert': #embeddings or encoder
447447
if toks[1] == 'encoder': #transformer
448-
448+
449449
if toks[-2] == 'layernorm': #bias->beta, weight->gamma
450450
toks[-1] = 'beta' if toks[-1] == 'bias' else 'gamma'
451451
elif (toks[-2] == 'dense' or toks[-2] in {'key', 'value', 'query'}) and toks[-1] == 'weight':
@@ -455,7 +455,7 @@ def onnx_to_trt_name(onnx_name):
455455
toks[-2] = 'kernel'
456456
elif toks[-2] == 'input_quantizer':
457457
toks[-2] = 'input'
458-
458+
459459
if 'final_input_quantizer' not in toks[2]:
460460
toks = toks[3:]
461461
toks[0] = 'l{}'.format(int(toks[0]))
@@ -503,10 +503,10 @@ def load_onnx_weights_and_quant(path, config):
503503
Bqkv[0,:] = tensor
504504
Bqkv[1,:] = tensor_dict[prefix + BK]
505505
Bqkv[2,:] = tensor_dict[prefix + BV]
506-
506+
507507
Wqkv = np.ascontiguousarray(Wqkv.reshape((3, N, H, N, H)).transpose((1,0,2,3,4)))
508508
Bqkv = np.ascontiguousarray(Bqkv.reshape((3, N, H)).transpose((1,0,2)))
509-
509+
510510
weights_dict[prefix + WQKV] = trt.Weights(Wqkv)
511511
weights_dict[prefix + BQKV] = trt.Weights(Bqkv)
512512
weights_dict[prefix + WQKV + "_notrans"] = trt.Weights(Wqkv.T)

‎demo/BERT/builder_varseqlen.py

+14-14
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#!/usr/bin/env python3
22
#
3-
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
3+
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
44
#
55
# Licensed under the Apache License, Version 2.0 (the "License");
66
# you may not use this file except in compliance with the License.
@@ -165,9 +165,9 @@ def attention_layer_opt(prefix, config, init_dict, network, input_tensor, mask_i
165165
qkv2ctx_plug = mha_plg_creator3.create_plugin("qkv2ctx", pfc)
166166
qkv_in = [mult_all.get_output(0), cu_seqlens, max_seqlen]
167167
else:
168-
fields.append(pf_has_mask)
169-
fields.append(pf_type)
170-
fields.append(pf_var_seqlen)
168+
fields.append(pf_has_mask)
169+
fields.append(pf_type)
170+
fields.append(pf_var_seqlen)
171171
pfc = trt.PluginFieldCollection(fields)
172172
qkv2ctx_plug = mha_plg_creator2.create_plugin("qkv2ctx", pfc)
173173
qkv_in = [mult_all.get_output(0), mask_idx, cu_seqlens, max_seqlen]
@@ -212,7 +212,7 @@ def transformer_layer_opt(prefix, config, init_dict, network, input_tensor, mask
212212
hidden_size = config.hidden_size
213213

214214
if config.use_qat:
215-
dr_input = init_dict[prefix + 'attention_self_query_input_amax']
215+
dr_input = init_dict[prefix + 'attention_self_query_input_amax']
216216
assert(dr_input ==init_dict[prefix + 'attention_self_key_input_amax'] )
217217
assert(dr_input ==init_dict[prefix + 'attention_self_value_input_amax'] )
218218
input_tensor.set_dynamic_range(-dr_input, dr_input)
@@ -270,7 +270,7 @@ def transformer_layer_opt(prefix, config, init_dict, network, input_tensor, mask
270270
dr_gelu = init_dict[prefix + 'output_dense_input_amax']
271271
set_output_range(gelu_layer, dr_gelu)
272272
else:
273-
# use gelu10 according to whitepaper http://arxiv.org/abs/2004.09602
273+
# use gelu10 according to whitepaper http://arxiv.org/abs/2004.09602
274274
set_output_range(gelu_layer, 10)
275275

276276
# FC2
@@ -423,7 +423,7 @@ def onnx_to_trt_name(onnx_name):
423423
toks = [t.strip('_') for t in onnx_name.split('.')]
424424
if toks[0] == 'bert': #embeddings or encoder
425425
if toks[1] == 'encoder': #transformer
426-
426+
427427
if toks[-2] == 'layernorm': #bias->beta, weight->gamma
428428
toks[-1] = 'beta' if toks[-1] == 'bias' else 'gamma'
429429
elif (toks[-2] == 'dense' or toks[-2] in {'key', 'value', 'query'}) and toks[-1] == 'weight':
@@ -433,7 +433,7 @@ def onnx_to_trt_name(onnx_name):
433433
toks[-2] = 'kernel'
434434
elif toks[-2] == 'input_quantizer':
435435
toks[-2] = 'input'
436-
436+
437437
if 'final_input_quantizer' not in toks[2]:
438438
toks = toks[3:]
439439
toks[0] = 'l{}'.format(int(toks[0]))
@@ -481,14 +481,14 @@ def load_onnx_weights_and_quant(path, config):
481481
Bqkv[0,:] = tensor
482482
Bqkv[1,:] = tensor_dict[prefix + BK]
483483
Bqkv[2,:] = tensor_dict[prefix + BV]
484-
484+
485485
if config.use_int8 and config.interleaved:
486486
Wqkv = np.ascontiguousarray(Wqkv.reshape((3, N, H, N, H)))
487487
Bqkv = np.ascontiguousarray(Bqkv.reshape((3, N, H)))
488488
else:
489489
Wqkv = np.ascontiguousarray(Wqkv.reshape((3, N, H, N, H)).transpose((1,0,2,3,4)))
490490
Bqkv = np.ascontiguousarray(Bqkv.reshape((3, N, H)).transpose((1,0,2)))
491-
491+
492492
weights_dict[prefix + WQKV] = trt.Weights(Wqkv)
493493
weights_dict[prefix + BQKV] = trt.Weights(Bqkv)
494494
weights_dict[prefix + WQKV + "_notrans"] = trt.Weights(Wqkv.T)
@@ -513,7 +513,7 @@ def emb_layernorm(builder, network, config, weights_dict, builder_config, max_se
513513
cu_seqlens = network.add_input(name="cu_seqlens", dtype=trt.int32, shape=(-1,))
514514
max_seqlen = network.add_input(name="max_seqlen", dtype=trt.int32, shape=(-1,))
515515

516-
# Specify profiles
516+
# Specify profiles
517517
profile = builder.create_optimization_profile()
518518
min_shape = (1,)
519519
shape = (max_sequence_length*max_batch_size,)
@@ -538,7 +538,7 @@ def emb_layernorm(builder, network, config, weights_dict, builder_config, max_se
538538
emb_layer = network.add_plugin_v2(inputs, fn)
539539

540540
if config.use_int8 and config.use_qat:
541-
dr_input = weights_dict['l0_attention_self_query_input_amax']
541+
dr_input = weights_dict['l0_attention_self_query_input_amax']
542542
set_output_range(emb_layer, dr_input)
543543
set_output_name(emb_layer, "embeddings_", "output")
544544
return emb_layer, cu_seqlens, max_seqlen
@@ -559,9 +559,9 @@ def build_engine(batch_size, workspace_size, sequence_length, config, weights_di
559559
emb_layer, cu_seqlens, max_seqlen = emb_layernorm(builder, network, config, weights_dict, builder_config, sequence_length, batch_size)
560560
embeddings = emb_layer.get_output(0)
561561
if config.use_int8 and config.interleaved:
562-
shuffle = network.add_shuffle(embeddings)
562+
shuffle = network.add_shuffle(embeddings)
563563
shuffle.second_transpose = (2, 1, 0, 3)
564-
embeddings = shuffle.get_output(0)
564+
embeddings = shuffle.get_output(0)
565565
mask_idx = None
566566
else:
567567
mask_idx = emb_layer.get_output(1)

‎demo/BERT/helpers/calibrator.py

+1-2
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#!/usr/bin/env python3
22
#
3-
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
3+
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
44
#
55
# Licensed under the Apache License, Version 2.0 (the "License");
66
# you may not use this file except in compliance with the License.
@@ -109,4 +109,3 @@ def read_histogram_cache(self, length):
109109

110110
def write_histogram_cache(self, ptr, length):
111111
return None
112-

0 commit comments

Comments
 (0)