Lint trailing newlines (pytorch#54737)

Summary:
*Context:* pytorch#53406 added a lint for trailing whitespace at the ends of lines. However, in order to pass FB-internal lints, that PR also had to normalize the trailing newlines in four of the files it touched. This PR adds an OSS lint to normalize trailing newlines.

The changes to the following files (made in 54847d0) are the only manually-written parts of this PR:

- `.github/workflows/lint.yml`
- `mypy-strict.ini`
- `tools/README.md`
- `tools/test/test_trailing_newlines.py`
- `tools/trailing_newlines.py`

I would have liked to make this just a shell one-liner like the other three similar lints, but nothing I could find quite fit the bill. Specifically, all the answers I tried from the following Stack Overflow questions were far too slow (at least a minute and a half to run on this entire repository):

- [How to detect file ends in newline?](https://stackoverflow.com/q/38746)
- [How do I find files that do not end with a newline/linefeed?](https://stackoverflow.com/q/4631068)
- [How to list all files in the Git index without newline at end of file](https://stackoverflow.com/q/27624800)
- [Linux - check if there is an empty line at the end of a file [duplicate]](https://stackoverflow.com/q/34943632)
- [git ensure newline at end of each file](https://stackoverflow.com/q/57770972)

To avoid giving false positives during the few days after this PR is merged, we should probably only merge it after pytorch#54967.

Pull Request resolved: pytorch#54737

Test Plan:
Running the shell script from the "Ensure correct trailing newlines" step in the `quick-checks` job of `.github/workflows/lint.yml` should print no output and exit in a fraction of a second with a status of 0. That was not the case prior to this PR, as shown by this failing GHA workflow run on an earlier draft of this PR:

- https://github.com/pytorch/pytorch/runs/2197446987?check_suite_focus=true

In contrast, this run (after correcting the trailing newlines in this PR) succeeded:

- https://github.com/pytorch/pytorch/pull/54737/checks?check_run_id=2197553241

To unit-test `tools/trailing_newlines.py` itself (this is run as part of our "Test tools" GitHub Actions workflow):
```
python tools/test/test_trailing_newlines.py
```

Reviewed By: malfet

Differential Revision: D27409736

Pulled By: samestep

fbshipit-source-id: 46f565227046b39f68349bbd5633105b2d2e9b19

.circleci/windows-jni/include/jni.h

@@ -1129,4 +1129,3 @@ JNIEXPORT void JNI_OnUnload(JavaVM* vm, void* reserved);
 #define JNI_ABORT       2           /* free buffer w/o copying back */
 
 #endif  /* JNI_H_ */
-

.github/workflows/lint.yml

@@ -30,6 +30,9 @@ jobs:
           rm -r "shellcheck-${scversion}"
           shellcheck --version
           .jenkins/run-shellcheck.sh
+      - name: Ensure correct trailing newlines
+        run: |
+          (! git grep -Il '' -- . ':(exclude)**/contrib/**' ':(exclude)third_party' ':(exclude)**.expect' | tools/trailing_newlines.py || (echo "The above files do not have correct trailing newlines; please normalize them"; false))
       - name: Ensure no trailing spaces
         run: |
           (! git grep -I -no ' $' -- . ':(exclude)**/contrib/**' ':(exclude)third_party' || (echo "The above files have trailing spaces; please remove them"; false))

.jenkins/pytorch/perf_test/test_cpu_speed_torch.sh

@@ -27,4 +27,3 @@ test_cpu_speed_torch () {
 if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
   run_test test_cpu_speed_torch "$@"
 fi
-

.jenkins/pytorch/perf_test/test_cpu_speed_torch_tensor.sh

@@ -27,4 +27,3 @@ test_cpu_speed_torch_tensor () {
 if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
   run_test test_cpu_speed_torch_tensor "$@"
 fi
-

android/test_app/app/src/main/res/layout/activity_main.xml

@@ -14,4 +14,4 @@
         android:background="@android:color/black"
         android:textColor="@android:color/white" />
 
-</FrameLayout>
+</FrameLayout>

aten/src/ATen/NumericUtils.h

@@ -105,4 +105,3 @@ C10_HOST_DEVICE inline double tan<double>(double x) {
 }
 
 } // namespace at
-

aten/src/ATen/cpu/vec256/vsx/vec256_complex_double_vsx.h

@@ -594,4 +594,3 @@ Vec256<ComplexDbl> inline minimum(
 } // namespace
 } // namespace vec256
 } // namespace at
-

aten/src/ATen/cpu/vec256/vsx/vsx_helpers.h

@@ -386,4 +386,3 @@ const vfloat64 vd_pi_2 = vfloat64{M_PI / 2.0, 0.0};
 } // namespace
 } // namespace vec256
 } // namespace at
-

aten/src/ATen/cudnn/cudnn-wrapper.h

@@ -13,4 +13,3 @@
 
 #undef STRINGIFY
 #undef STRING
-

aten/src/ATen/div_rtn.h

@@ -8,4 +8,3 @@ static inline T div_rtn(T x, T y) {
     if ((r!=0) && ((r<0) != (y<0))) --q;
     return q;
 }
-

aten/src/ATen/native/Cross.h

@@ -10,4 +10,3 @@ using cross_fn = void(*)(Tensor&, const Tensor&, const Tensor&, const int64_t d)
 DECLARE_DISPATCH(cross_fn, cross_stub);
 
 }} // namespace at::native
-

aten/src/ATen/native/batch_norm.h

@@ -15,4 +15,3 @@ DECLARE_DISPATCH(batch_norm_fn, batch_norm_cpu_inference_contiguous_stub);
 } // namespace native
 
 } // namespace at
-

aten/src/ATen/native/cpu/CrossKernel.cpp

@@ -75,4 +75,3 @@ static void cross_kernel_impl(Tensor& result, const Tensor& a, const Tensor& b,
 REGISTER_DISPATCH(cross_stub, &cross_kernel_impl);
 
 }} // namespace at::native
-

aten/src/ATen/native/cuda/CrossKernel.cu

@@ -12,4 +12,3 @@ void cross_kernel_impl(Tensor& result, const Tensor& x1, const Tensor& x2, const
 REGISTER_DISPATCH(cross_stub, &cross_kernel_impl);
 
 }}
-

aten/src/ATen/native/cuda/EmbeddingBackwardKernel.cuh

@@ -33,4 +33,4 @@ Tensor embedding_backward_cuda_kernel(
     const Tensor &bag_size = Tensor(),
     const Tensor &per_sample_weights = Tensor());
 
-}}
+}}

aten/src/ATen/native/quantized/cpu/kernels/README.md

@@ -13,4 +13,4 @@
     IS THE CASE CAN LEAD TO HARD-TO-DEBUG ODR VIOLATIONS.
   - **Make sure different variants of the code (AVX, AVX2) are tested!**
     There are build variants that do things like have NO AVX and NO AVX2 in
-    CI. Make sure they work!
+    CI. Make sure they work!

aten/src/ATen/native/quantized/cpu/qnnpack/CONTRIBUTING.md

@@ -31,4 +31,4 @@ outlined on that page and do not file a public issue.
 
 ## License
 By contributing to QNNPACK, you agree that your contributions will be licensed
-under the LICENSE file in the root directory of this source tree.
+under the LICENSE file in the root directory of this source tree.

aten/src/ATen/test/NamedTensor_test.cpp

@@ -233,5 +233,3 @@ TEST(NamedTensorTest, TensorNamesCheckUnique) {
     ASSERT_THROW(tensornames.checkUnique("op_name"), c10::Error);
   }
 }
-
-

aten/src/ATen/test/cuda_complex_math_test.cu

@@ -35,4 +35,3 @@ __global__ void CUDA##a##b()
 #define C10_DEFINE_TEST(a, b) TEST(a##Host, b)
 #define C10_ASSERT_NEAR(a, b, tol) ASSERT_NEAR(a, b, tol)
 #include <c10/test/util/complex_math_test_common.h>
-

aten/src/ATen/test/pow_test.cpp

@@ -364,4 +364,3 @@ TEST(PowTest, TestIntegralPow) {
   test_inverse(longs);
   test_inverse(ints);
 }
-

aten/src/THC/generated/THCTensorMathCompareTBool.cu

@@ -1,3 +1,2 @@
 #include <THC/THCTensorMathCompareT.cuh>
 #include <THC/THCTensor.hpp>
-

aten/src/THC/generated/THCTensorMathCompareTByte.cu

@@ -1,3 +1,2 @@
 #include <THC/THCTensorMathCompareT.cuh>
 #include <THC/THCTensor.hpp>
-

aten/src/THC/generated/THCTensorMathCompareTChar.cu

@@ -1,3 +1,2 @@
 #include <THC/THCTensorMathCompareT.cuh>
 #include <THC/THCTensor.hpp>
-

aten/src/THC/generated/THCTensorMathCompareTDouble.cu

@@ -1,3 +1,2 @@
 #include <THC/THCTensorMathCompareT.cuh>
 #include <THC/THCTensor.hpp>
-

aten/src/THC/generated/THCTensorMathCompareTFloat.cu

@@ -1,3 +1,2 @@
 #include <THC/THCTensorMathCompareT.cuh>
 #include <THC/THCTensor.hpp>
-

aten/src/THC/generated/THCTensorMathCompareTHalf.cu

@@ -1,3 +1,2 @@
 #include <THC/THCTensorMathCompareT.cuh>
 #include <THC/THCTensor.hpp>
-

aten/src/THC/generated/THCTensorMathCompareTInt.cu

@@ -1,3 +1,2 @@
 #include <THC/THCTensorMathCompareT.cuh>
 #include <THC/THCTensor.hpp>
-

aten/src/THC/generated/THCTensorMathCompareTLong.cu

@@ -1,3 +1,2 @@
 #include <THC/THCTensorMathCompareT.cuh>
 #include <THC/THCTensor.hpp>
-

aten/src/THC/generated/THCTensorMathCompareTShort.cu

@@ -1,3 +1,2 @@
 #include <THC/THCTensorMathCompareT.cuh>
 #include <THC/THCTensor.hpp>
-

benchmarks/README.md

@@ -26,4 +26,3 @@ python -c "import torch; print(torch.__version__)"
 Please refer to each subfolder to discover each benchmark suite
 
 * [Fast RNNs benchmarks](fastrnns/README.md)
-

benchmarks/fastrnns/README.md

@@ -39,4 +39,3 @@ you can also specify the models to generate nvprof files separately:
 
 Use Linux for the most accurate timing. A lot of these tests only run
 on CUDA.
-

c10/util/Optional.cpp

@@ -7,4 +7,4 @@
 #if (!defined(__CUDA_ARCH__) || !defined(CUDA_VERSION) || CUDA_VERSION > 9200)
 static_assert(C10_IS_TRIVIALLY_COPYABLE(c10::optional<int>), "c10::optional<int> should be trivially copyable");
 static_assert(C10_IS_TRIVIALLY_COPYABLE(c10::optional<bool>), "c10::optional<bool> should be trivially copyable");
-#endif
+#endif

c10/util/Unicode.cpp

@@ -46,4 +46,4 @@ std::string u16u8(const std::wstring& wstr) {
     return str;
 }
 #endif
-}  // namespace c10
+}  // namespace c10

caffe2/core/common_test.cc

@@ -41,5 +41,3 @@ TEST(CommonTest, TestStod) {
 #endif // __ANDROID__
 
 }  // namespace caffe2
-
-

caffe2/core/nomnigraph/tests/AlgorithmsTest.cc

@@ -123,4 +123,3 @@ TEST(Subgraph, InduceEdgesCycle) {
     EXPECT_TRUE(sg.hasEdge(edge));
   }
 }
-

caffe2/onnx/device.cc

@@ -15,4 +15,3 @@ Device::Device(const std::string &spec) {
   device_id = atoi(spec.substr(pos + 1).c_str());
 }
 }}
-

caffe2/operators/roi_align_op.cc

@@ -312,4 +312,3 @@ C10_EXPORT_CAFFE2_OP_TO_C10_CPU(
     "    bool aligned"
     ") -> Tensor",
     caffe2::RoIAlignCPUOp<float>);
-

caffe2/opt/optimizer.cc

@@ -45,4 +45,3 @@ NetDef optimize(NetDef net, int level) {
 
 } // namespace opt
 } // namespace caffe2
-

caffe2/python/CMakeLists.txt

@@ -33,4 +33,3 @@ prepend(Caffe2_HIP_PYTHON_SRCS ${CMAKE_CURRENT_SOURCE_DIR} ${Caffe2_HIP_PYTHON_S
 set(Caffe2_CPU_PYTHON_SRCS ${Caffe2_CPU_PYTHON_SRCS} PARENT_SCOPE)
 set(Caffe2_GPU_PYTHON_SRCS ${Caffe2_GPU_PYTHON_SRCS} PARENT_SCOPE)
 set(Caffe2_HIP_PYTHON_SRCS ${Caffe2_HIP_PYTHON_SRCS} PARENT_SCOPE)
-

caffe2/python/convert.py

@@ -1,7 +1,2 @@
 ## @package workspace
 # Module caffe2.python.workspace
-
-
-
-
-

caffe2/python/fakelowp/init_shared_libs.py

@@ -7,4 +7,3 @@
     lib = os.environ['OSS_ONNXIFI_LIB']
     print("Loading ONNXIFI lib: ".format(lib))
     ctypes.CDLL(lib, ctypes.RTLD_GLOBAL)
-

caffe2/python/onnx/README.md

@@ -51,4 +51,3 @@ pip install -e caffe2/
 # License
 
 [MIT License](LICENSE)
-

caffe2/python/onnx/backend_cpp_rep.py

@@ -49,4 +49,3 @@ def run(self, inputs):
             # single input
             output_values = self.__core.run([inputs])
         return namedtupledict('Outputs', self.__external_outputs)(*output_values)
-

caffe2/python/onnx/test_onnxifi.py

@@ -197,5 +197,3 @@ def test_resnet50_core(self):
             output_values = [workspace.FetchBlob(name) for name in net_outputs]
             Y_trt = namedtupledict('Outputs', net_outputs)(*output_values)
         np.testing.assert_allclose(Y_c2, Y_trt, rtol=1e-3)
-
-

caffe2/python/onnx/tests/__init__.py

@@ -1,4 +0,0 @@
-
-
-
-

caffe2/python/onnx/tests/test_utils.py

@@ -27,4 +27,3 @@ def add_test_case(self, name, test_func):
         if hasattr(self, name):
             raise ValueError('Duplicated test name: {}'.format(name))
         setattr(self, name, test_func)
-

caffe2/python/operator_test/mean_op_test.py

@@ -60,4 +60,3 @@ def mean_ref(*args):
 
 if __name__ == "__main__":
     unittest.main()
-

caffe2/python/rnn/__init__.py

@@ -1,5 +0,0 @@
-
-
-
-
-

caffe2/python/serialized_test/SerializedTestCoverage.md

@@ -876,4 +876,3 @@ Serialized tests have covered 220/852 (25.8%) operators
 * LengthsSumFakeFp16
 * SparseLengthsMax
 </details>
-

caffe2/python/trt/data/class_labels.txt

@@ -997,4 +997,4 @@ earthstar
 hen-of-the-woods
 bolete
 ear
-toilet tissue
+toilet tissue

caffe2/python/trt/test_trt.py

@@ -277,5 +277,3 @@ def test_resnet50_core(self):
             output_values = [workspace.FetchBlob(name) for name in net_outputs]
             Y_trt = namedtupledict('Outputs', net_outputs)(*output_values)
         np.testing.assert_allclose(Y_c2, Y_trt, rtol=1e-3)
-
-

caffe2/python/trt/transform.py

@@ -106,4 +106,3 @@ def transform_caffe2_net(
     pred_net_cut = caffe2_pb2.NetDef()
     pred_net_cut.ParseFromString(pred_net_str)
     return pred_net_cut
-

caffe2/utils/simple_queue_test.cc

@@ -68,5 +68,3 @@ TEST(SimpleQueueDeathTest, CannotAddAfterQueueFinished) {
 
 
 }  // namespace caffe2
-
-

cmake/Modules/FindAtlas.cmake

@@ -49,4 +49,3 @@ if(ATLAS_FOUND)
 
   message(STATUS "Found Atlas (include: ${Atlas_CBLAS_INCLUDE_DIR}, library: ${Atlas_BLAS_LIBRARY})")
 endif(ATLAS_FOUND)
-

cmake/Modules/FindHiredis.cmake

@@ -21,4 +21,3 @@ if(HIREDIS_FOUND)
   message(STATUS "Found Hiredis  (include: ${Hiredis_INCLUDE_DIR}, library: ${Hiredis_LIBRARIES})")
   mark_as_advanced(Hiredis_INCLUDE_DIR Hiredis_LIBRARIES)
 endif()
-

cmake/Modules/FindNumPy.cmake

@@ -55,4 +55,3 @@ if(NUMPY_FOUND)
 endif()
 
 caffe_clear_vars(__result __output __error_value __values __ver_check __error_value)
-

cmake/Modules/FindNuma.cmake

@@ -26,4 +26,3 @@ if(NUMA_FOUND)
       "Found Numa  (include: ${Numa_INCLUDE_DIR}, library: ${Numa_LIBRARIES})")
   mark_as_advanced(Numa_INCLUDE_DIR Numa_LIBRARIES)
 endif()
-

cmake/Modules/FindOpenBLAS.cmake

@@ -63,4 +63,3 @@ MARK_AS_ADVANCED(
     OpenBLAS_LIB
     OpenBLAS
 )
-

cmake/Modules/FindRocksDB.cmake

@@ -21,4 +21,3 @@ if(ROCKSDB_FOUND)
   message(STATUS "Found RocksDB  (include: ${RocksDB_INCLUDE_DIR}, library: ${RocksDB_LIBRARIES})")
   mark_as_advanced(RocksDB_INCLUDE_DIR RocksDB_LIBRARIES)
 endif()
-

cmake/Modules/FindSnappy.cmake

@@ -24,4 +24,3 @@ if(SNAPPY_FOUND)
                      SNAPPY_VERION_LINES SNAPPY_MAJOR SNAPPY_MINOR SNAPPY_PATCHLEVEL)
   set(Snappy_VERSION "${SNAPPY_MAJOR}.${SNAPPY_MINOR}.${SNAPPY_PATCHLEVEL}")
 endif()
-

cmake/Modules/FindVSX.cmake

@@ -32,4 +32,3 @@ IF(CMAKE_SYSTEM_NAME MATCHES "Linux")
   endif()
   message("-- </FindVSX>")
 endif()
-

cmake/Modules_CUDA_fix/upstream/FindCUDA/parse_cubin.cmake

@@ -107,5 +107,3 @@ if (NOT "${file_text}" STREQUAL "")
 else()
   # message("FOUND NO DEPENDS")
 endif()
-
-

cmake/public/gflags.cmake

@@ -81,4 +81,3 @@ if(NOT TARGET gflags)
       "Caffe2 or a Caffe2 dependent library, the next warning / error will "
       "give you more info.")
 endif()
-

cmake/public/glog.cmake

@@ -68,4 +68,3 @@ if(NOT TARGET glog::glog)
       "Caffe2 or a Caffe2 dependent library, the next warning / error will "
       "give you more info.")
 endif()
-

cmake/public/utils.cmake

@@ -484,4 +484,3 @@ function(torch_set_target_props libname)
     set_target_properties(${libname} PROPERTIES STATIC_LIBRARY_FLAGS_DEBUG "/NODEFAULTLIB:${VCOMP_LIB}d")
   endif()
 endfunction()
-

docs/caffe2/DOXYGEN.md

@@ -71,4 +71,3 @@ def my_method(x, y):"""!
 ```
 
 Note that the bang (!) is added after the opening comment """! - this seems to do the trick and the remaining comments will be nicely parsed by Doxygen.
-

docs/cpp/Makefile

@@ -25,4 +25,3 @@ clean:
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile
 	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
-

docs/source/community/persons_of_interest.rst

@@ -179,6 +179,3 @@ TorchAudio
 ~~~~~~~~~~
 
 - Vincent QB (`vincentqb <https://github.com/vincentqb>`__)
-
-
-

docs/source/dlpack.rst

@@ -5,4 +5,3 @@ torch.utils.dlpack
 
 .. autofunction:: from_dlpack
 .. autofunction:: to_dlpack
-

docs/source/jit_builtin_functions.rst

@@ -8,4 +8,3 @@ This is a full reference of functions and Tensor methods accessible in TorchScri
 .. contents:: :local:
 
 .. automodule:: torch.jit.supported_ops
-

docs/source/notes/randomness.rst

@@ -149,5 +149,3 @@ Use :meth:`worker_init_fn` to preserve reproducibility::
         num_workers=num_workers,
         worker_init_fn=seed_worker
     )
-
-

docs/source/special.rst

@@ -24,4 +24,4 @@ Functions
 .. autofunction:: erfinv
 .. autofunction:: expm1
 .. autofunction:: exp2
-.. autofunction:: gammaln
+.. autofunction:: gammaln

docs/source/storage.rst

@@ -85,4 +85,3 @@ Every :class:`torch.Tensor` has a corresponding storage of the same data type.
    :members:
    :undoc-members:
    :inherited-members:
-