forked from pytorch/pytorch.github.io
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathcontribution_guide.html
1476 lines (1295 loc) · 85.8 KB
/
contribution_guide.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>PyTorch Contribution Guide — PyTorch master documentation</title>
<link rel="canonical" href="https://pytorch.org/docs/stable/community/contribution_guide.html"/>
<link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
<!-- <link rel="stylesheet" href="../_static/pygments.css" type="text/css" /> -->
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/dist/katex.min.css" type="text/css" />
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/dist/katex.min.css" type="text/css" />
<link rel="stylesheet" href="../_static/katex-math.css" type="text/css" />
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="PyTorch Governance" href="governance.html" />
<link rel="prev" title="Windows FAQ" href="../notes/windows.html" />
<script src="../_static/js/modernizr.min.js"></script>
</head>
<div class="container-fluid header-holder tutorials-header" id="header-holder">
<div class="container">
<div class="header-container">
<a class="header-logo" href="https://pytorch.org/" aria-label="PyTorch"></a>
<div class="main-menu">
<ul>
<li>
<a href="https://pytorch.org/get-started">Get Started</a>
</li>
<li>
<a href="https://pytorch.org/features">Features</a>
</li>
<li>
<a href="https://pytorch.org/ecosystem">Ecosystem</a>
</li>
<li>
<a href="https://pytorch.org/blog/">Blog</a>
</li>
<li>
<a href="https://pytorch.org/tutorials">Tutorials</a>
</li>
<li class="active">
<a href="https://pytorch.org/docs/stable/index.html">Docs</a>
</li>
<li>
<a href="https://pytorch.org/resources">Resources</a>
</li>
<li>
<a href="https://github.com/pytorch/pytorch">Github</a>
</li>
</ul>
</div>
<a class="main-menu-open-button" href="#" data-behavior="open-mobile-menu"></a>
</div>
</div>
</div>
<body class="pytorch-body">
<div class="table-of-contents-link-wrapper">
<span>Table of Contents</span>
<a href="#" class="toggle-table-of-contents" data-behavior="toggle-table-of-contents"></a>
</div>
<nav data-toggle="wy-nav-shift" class="pytorch-left-menu" id="pytorch-left-menu">
<div class="pytorch-side-scroll">
<div class="pytorch-menu pytorch-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<div class="pytorch-left-menu-search">
<div class="version">
<a href='http://pytorch.org/docs/versions.html'>1.1.0a0+654e59f ▼</a>
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
<input type="text" name="q" placeholder="Search Docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div>
<a style="color:#F05732" href="https://pytorch.org/docs/stable/community/contribution_guide.html">
You are viewing unstable developer preview docs.
Click here to view docs for latest stable release.
</a>
</div>
<p class="caption"><span class="caption-text">Notes</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../notes/autograd.html">Autograd mechanics</a></li>
<li class="toctree-l1"><a class="reference internal" href="../notes/broadcasting.html">Broadcasting semantics</a></li>
<li class="toctree-l1"><a class="reference internal" href="../notes/cuda.html">CUDA semantics</a></li>
<li class="toctree-l1"><a class="reference internal" href="../notes/extending.html">Extending PyTorch</a></li>
<li class="toctree-l1"><a class="reference internal" href="../notes/faq.html">Frequently Asked Questions</a></li>
<li class="toctree-l1"><a class="reference internal" href="../notes/multiprocessing.html">Multiprocessing best practices</a></li>
<li class="toctree-l1"><a class="reference internal" href="../notes/randomness.html">Reproducibility</a></li>
<li class="toctree-l1"><a class="reference internal" href="../notes/serialization.html">Serialization semantics</a></li>
<li class="toctree-l1"><a class="reference internal" href="../notes/windows.html">Windows FAQ</a></li>
</ul>
<p class="caption"><span class="caption-text">Community</span></p>
<ul class="current">
<li class="toctree-l1 current"><a class="current reference internal" href="#">PyTorch Contribution Guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="governance.html">PyTorch Governance</a></li>
<li class="toctree-l1"><a class="reference internal" href="persons_of_interest.html">PyTorch Governance | Persons of Interest</a></li>
</ul>
<p class="caption"><span class="caption-text">Package Reference</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../torch.html">torch</a></li>
<li class="toctree-l1"><a class="reference internal" href="../tensors.html">torch.Tensor</a></li>
<li class="toctree-l1"><a class="reference internal" href="../tensor_attributes.html">Tensor Attributes</a></li>
<li class="toctree-l1"><a class="reference internal" href="../type_info.html">Type Info</a></li>
<li class="toctree-l1"><a class="reference internal" href="../sparse.html">torch.sparse</a></li>
<li class="toctree-l1"><a class="reference internal" href="../cuda.html">torch.cuda</a></li>
<li class="toctree-l1"><a class="reference internal" href="../storage.html">torch.Storage</a></li>
<li class="toctree-l1"><a class="reference internal" href="../nn.html">torch.nn</a></li>
<li class="toctree-l1"><a class="reference internal" href="../nn.html#torch-nn-functional">torch.nn.functional</a></li>
<li class="toctree-l1"><a class="reference internal" href="../nn.html#torch-nn-init">torch.nn.init</a></li>
<li class="toctree-l1"><a class="reference internal" href="../optim.html">torch.optim</a></li>
<li class="toctree-l1"><a class="reference internal" href="../autograd.html">torch.autograd</a></li>
<li class="toctree-l1"><a class="reference internal" href="../distributed.html">torch.distributed</a></li>
<li class="toctree-l1"><a class="reference internal" href="../distributions.html">torch.distributions</a></li>
<li class="toctree-l1"><a class="reference internal" href="../jit.html">torch.jit</a></li>
<li class="toctree-l1"><a class="reference internal" href="../multiprocessing.html">torch.multiprocessing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../bottleneck.html">torch.utils.bottleneck</a></li>
<li class="toctree-l1"><a class="reference internal" href="../checkpoint.html">torch.utils.checkpoint</a></li>
<li class="toctree-l1"><a class="reference internal" href="../cpp_extension.html">torch.utils.cpp_extension</a></li>
<li class="toctree-l1"><a class="reference internal" href="../data.html">torch.utils.data</a></li>
<li class="toctree-l1"><a class="reference internal" href="../dlpack.html">torch.utils.dlpack</a></li>
<li class="toctree-l1"><a class="reference internal" href="../hub.html">torch.hub</a></li>
<li class="toctree-l1"><a class="reference internal" href="../model_zoo.html">torch.utils.model_zoo</a></li>
<li class="toctree-l1"><a class="reference internal" href="../onnx.html">torch.onnx</a></li>
<li class="toctree-l1"><a class="reference internal" href="../distributed_deprecated.html">torch.distributed.deprecated</a></li>
</ul>
<p class="caption"><span class="caption-text">torchvision Reference</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../torchvision/index.html">torchvision</a></li>
</ul>
</div>
</div>
</nav>
<div class="pytorch-container">
<div class="pytorch-page-level-bar" id="pytorch-page-level-bar">
<div class="pytorch-breadcrumbs-wrapper">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="pytorch-breadcrumbs">
<li>
<a href="../index.html">
Docs
</a> >
</li>
<li>PyTorch Contribution Guide</li>
<li class="pytorch-breadcrumbs-aside">
<a href="../_sources/community/contribution_guide.rst.txt" rel="nofollow"><img src="../_static/images/view-page-source-icon.svg"></a>
</li>
</ul>
</div>
</div>
<div class="pytorch-shortcuts-wrapper" id="pytorch-shortcuts-wrapper">
Shortcuts
</div>
</div>
<section data-toggle="wy-nav-shift" id="pytorch-content-wrap" class="pytorch-content-wrap">
<div class="pytorch-content-left">
<div class="rst-content">
<div role="main" class="main-content" itemscope="itemscope" itemtype="http://schema.org/Article">
<article itemprop="articleBody" id="pytorch-article" class="pytorch-article">
<div class="section" id="pytorch-contribution-guide">
<h1>PyTorch Contribution Guide<a class="headerlink" href="#pytorch-contribution-guide" title="Permalink to this headline">¶</a></h1>
<p>PyTorch is a GPU-accelerated Python tensor computation package for
building deep neural networks built on tape-based autograd systems.</p>
<div class="section" id="the-pytorch-contribution-process">
<h2>The PyTorch Contribution Process<a class="headerlink" href="#the-pytorch-contribution-process" title="Permalink to this headline">¶</a></h2>
<p>The PyTorch organization is governed by <a class="reference external" href="/docs/community/governance.html">PyTorch
Governance</a>.</p>
<p>The PyTorch development process involves a healthy amount of open
discussions between the core development team and the community.</p>
<p>PyTorch operates similar to most open source projects on GitHub.
However, if you’ve never contributed to an open source project before,
here is the basic process.</p>
<ul class="simple">
<li><strong>Figure out what you’re going to work on.</strong> The majority of open
source contributions come from people scratching their own itches.
However, if you don’t know what you want to work on, or are just
looking to get more acquainted with the project, here are some tips
for how to find appropriate tasks:<ul>
<li>Look through the <a class="reference external" href="https://github.com/pytorch/pytorch/issues/">issue
tracker</a> and see if
there are any issues you know how to fix. Issues that are
confirmed by other contributors tend to be better to investigate.
We also maintain some labels for issues which are likely to be
good for new people, e.g., <strong>bootcamp</strong> and <strong>1hr</strong>, although
these labels are less well maintained.</li>
<li>Join us on Slack and let us know you’re interested in getting to
know PyTorch. We’re very happy to help out researchers and
partners get up to speed with the codebase.</li>
</ul>
</li>
<li><strong>Figure out the scope of your change and reach out for design
comments on a GitHub issue if it’s large.</strong> The majority of pull
requests are small; in that case, no need to let us know about what
you want to do, just get cracking. But if the change is going to be
large, it’s usually a good idea to get some design comments about it
first.<ul>
<li>If you don’t know how big a change is going to be, we can help you
figure it out! Just post about it on issues or Slack.</li>
<li>Some feature additions are very standardized; for example, lots of
people add new operators or optimizers to PyTorch. Design
discussion in these cases boils down mostly to, “Do we want this
operator/optimizer?” Giving evidence for its utility, e.g., usage
in peer reviewed papers, or existence in other frameworks, helps a
bit when making this case.</li>
<li>Core changes and refactors can be quite difficult to coordinate,
as the pace of development on PyTorch master is quite fast.
Definitely reach out about fundamental or cross-cutting changes;
we can often give guidance about how to stage such changes into
more easily reviewable pieces.</li>
</ul>
</li>
<li><strong>Code it out!</strong><ul>
<li>See the technical guide for advice for working with PyTorch in a
technical form.</li>
</ul>
</li>
<li><strong>Open a pull request.</strong><ul>
<li>If you are not ready for the pull request to be reviewed, tag it
with [WIP]. We will ignore it when doing review passes. If you are
working on a complex change, it’s good to start things off as WIP,
because you will need to spend time looking at CI results to see
if things worked out or not.</li>
<li>Find an appropriate reviewer for your change. We have some folks
who regularly go through the PR queue and try to review
everything, but if you happen to know who the maintainer for a
given subsystem affected by your patch is, feel free to include
them directly on the pull request. You can learn more about this
structure at PyTorch Subsystem Ownership.</li>
</ul>
</li>
<li><strong>Iterate on the pull request until it’s accepted!</strong><ul>
<li>We’ll try our best to minimize the number of review roundtrips and
block PRs only when there are major issues. For the most common
issues in pull requests, take a look at <a class="reference external" href="#common-mistakes-to-avoid">Common Mistakes</a>.</li>
<li>Once a pull request is accepted and CI is passing, there is
nothing else you need to do; we will merge the PR for you.</li>
</ul>
</li>
</ul>
</div>
<div class="section" id="getting-started">
<h2>Getting Started<a class="headerlink" href="#getting-started" title="Permalink to this headline">¶</a></h2>
<div class="section" id="proposing-new-features">
<h3>Proposing new features<a class="headerlink" href="#proposing-new-features" title="Permalink to this headline">¶</a></h3>
<p>New feature ideas are best discussed on a specific issue. Please include
as much information as you can, any accompanying data, and your proposed
solution. The PyTorch team and community frequently reviews new issues
and comments where they think they can help. If you feel confident in
your solution, go ahead and implement it.</p>
</div>
<div class="section" id="reporting-issues">
<h3>Reporting Issues<a class="headerlink" href="#reporting-issues" title="Permalink to this headline">¶</a></h3>
<p>If you’ve identified an issue, first search through the <a class="reference external" href="https://github.com/pytorch/pytorch/issues">list of
existing issues</a> on the
repo. If you are unable to find a similar issue, then create a new one.
Supply as much information you can to reproduce the problematic
behavior. Also, include any additional insights like the behavior you
expect.</p>
</div>
<div class="section" id="implementing-features-or-fixing-bugs">
<h3>Implementing Features or Fixing Bugs<a class="headerlink" href="#implementing-features-or-fixing-bugs" title="Permalink to this headline">¶</a></h3>
<p>If you want to fix a specific issue, it’s best to comment on the
individual issue with your intent. However, we do not lock or assign
issues except in cases where we have worked with the developer before.
It’s best to strike up a conversation on the issue and discuss your
proposed solution. The PyTorch team can provide guidance that saves you
time.</p>
<p>Issues that are labeled first-new-issue, low, or medium priority provide
the best entrance point are great places to start.</p>
</div>
<div class="section" id="adding-tutorials">
<h3>Adding Tutorials<a class="headerlink" href="#adding-tutorials" title="Permalink to this headline">¶</a></h3>
<p>A great deal of the tutorials on <a class="reference external" href="http://pytorch.org/">pytorch.org</a>
come from the community itself and we welcome additional contributions.
To learn more about how to contribute a new tutorial you can learn more
here: <a class="reference external" href="https://github.com/pytorch/tutorials/#contributing">PyTorch.org Tutorial Contribution Guide on
Github</a></p>
</div>
<div class="section" id="improving-documentation-tutorials">
<h3>Improving Documentation & Tutorials<a class="headerlink" href="#improving-documentation-tutorials" title="Permalink to this headline">¶</a></h3>
<p>We aim to produce high quality documentation and tutorials. On rare
occasions that content includes typos or bugs. If you find something you
can fix, send us a pull request for consideration.</p>
<p>Take a look at the <a class="reference external" href="#on-documentation">Documentation</a> section to learn how our system
works.</p>
</div>
<div class="section" id="participating-in-online-discussions">
<h3>Participating in online discussions<a class="headerlink" href="#participating-in-online-discussions" title="Permalink to this headline">¶</a></h3>
<p>You can find active discussions happening on the PyTorch Discussion
<a class="reference external" href="https://discuss.pytorch.org/">forum</a>.</p>
</div>
<div class="section" id="submitting-pull-requests-to-fix-open-issues">
<h3>Submitting pull requests to fix open issues<a class="headerlink" href="#submitting-pull-requests-to-fix-open-issues" title="Permalink to this headline">¶</a></h3>
<p>You can view a list of all open issues
<a class="reference external" href="https://github.com/pytorch/pytorch/issues">here</a>. Commenting on an
issue is a great way to get the attention of the team. From here you can
share your ideas and how you plan to resolve the issue.</p>
<p>For more challenging issues, the team will provide feedback and
direction for how to best solve the issue.</p>
<p>If you’re not able to fix the issue itself, commenting and sharing
whether you can reproduce the issue can be useful for helping the team
identify problem areas.</p>
</div>
<div class="section" id="reviewing-open-pull-requests">
<h3>Reviewing open pull requests<a class="headerlink" href="#reviewing-open-pull-requests" title="Permalink to this headline">¶</a></h3>
<p>We appreciate your help reviewing and commenting on pull requests. Our
team strives to keep the number of open pull requests at a manageable
size, we respond quickly for more information if we need it, and we
merge PRs that we think are useful. However, due to the high level of
interest, additional eyes on pull requests is appreciated.</p>
</div>
<div class="section" id="improving-code-readability">
<h3>Improving code readability<a class="headerlink" href="#improving-code-readability" title="Permalink to this headline">¶</a></h3>
<p>Improve code readability helps everyone. It is often better to submit a
small number of pull requests that touch few files versus a large pull
request that touches many files. Starting a discussion in the PyTorch
forum <a class="reference external" href="https://discuss.pytorch.org/">here</a> or on an issue related to
your improvement is the best way to get started.</p>
</div>
<div class="section" id="adding-test-cases-to-make-the-codebase-more-robust">
<h3>Adding test cases to make the codebase more robust<a class="headerlink" href="#adding-test-cases-to-make-the-codebase-more-robust" title="Permalink to this headline">¶</a></h3>
<p>Additional test coverage is appreciated.</p>
</div>
<div class="section" id="promoting-pytorch">
<h3>Promoting PyTorch<a class="headerlink" href="#promoting-pytorch" title="Permalink to this headline">¶</a></h3>
<p>Your use of PyTorch in your projects, research papers, write ups, blogs,
or general discussions around the internet helps to raise awareness for
PyTorch and our growing community. Please reach out to
<a class="reference external" href="http://mailto:pytorch-marketing@fb.com/">pytorch-marketing@fb.com</a>
for marketing support.</p>
</div>
<div class="section" id="triaging-issues">
<h3>Triaging issues<a class="headerlink" href="#triaging-issues" title="Permalink to this headline">¶</a></h3>
<p>If you feel that an issue could benefit from a particular tag or level
of complexity comment on the issue and share your opinion. If an you
feel an issue isn’t categorized properly comment and let the team know.</p>
</div>
</div>
<div class="section" id="about-open-source-development">
<h2>About open source development<a class="headerlink" href="#about-open-source-development" title="Permalink to this headline">¶</a></h2>
<p>If this is your first time contributing to an open source project, some
aspects of the development process may seem unusual to you.</p>
<ul class="simple">
<li><strong>There is no way to “claim” issues.</strong> People often want to “claim”
an issue when they decide to work on it, to ensure that there isn’t
wasted work when someone else ends up working on it. This doesn’t
really work too well in open source, since someone may decide to work
on something, and end up not having time to do it. Feel free to give
information in an advisory fashion, but at the end of the day, we
will take running code and rough consensus.</li>
<li><strong>There is a high bar for new functionality that is added.</strong> Unlike
in a corporate environment, where the person who wrote code
implicitly “owns” it and can be expected to take care of it in the
beginning of its lifetime, once a pull request is merged into an open
source project, it immediately becomes the collective responsibility
of all maintainers on the project. When we merge code, we are saying
that we, the maintainers, are able to review subsequent changes and
make a bugfix to the code. This naturally leads to a higher standard
of contribution.</li>
</ul>
</div>
<div class="section" id="common-mistakes-to-avoid">
<h2>Common Mistakes To Avoid<a class="headerlink" href="#common-mistakes-to-avoid" title="Permalink to this headline">¶</a></h2>
<ul class="simple">
<li><strong>Did you add tests?</strong> (Or if the change is hard to test, did you
describe how you tested your change?)<ul>
<li>We have a few motivations for why we ask for tests:<ol class="arabic">
<li>to help us tell if we break it later</li>
<li>to help us tell if the patch is correct in the first place
(yes, we did review it, but as Knuth says, “beware of the
following code, for I have not run it, merely proven it
correct”)</li>
</ol>
</li>
<li>When is it OK not to add a test? Sometimes a change can’t be
conveniently tested, or the change is so obviously correct (and
unlikely to be broken) that it’s OK not to test it. On the
contrary, if a change is seems likely (or is known to be likely)
to be accidentally broken, it’s important to put in the time to
work out a testing strategy.</li>
</ul>
</li>
<li><strong>Is your PR too long?</strong><ul>
<li>It’s easier for us to review and merge small PRs. Difficulty of
reviewing a PR scales nonlinearly with its size.</li>
<li>When is it OK to submit a large PR? It helps a lot if there was a
corresponding design discussion in an issue, with sign off from
the people who are going to review your diff. We can also help
give advice about how to split up a large change into individually
shippable parts. Similarly, it helps if there is a complete
description of the contents of the PR: it’s easier to review code
if we know what’s inside!</li>
</ul>
</li>
<li><strong>Comments for subtle things?</strong> In cases where behavior of your code
is nuanced, please include extra comments and documentation to allow
us to better understand the intention of your code.</li>
<li><strong>Did you add a hack?</strong> Sometimes a hack is the right answer. But
usually we will have to discuss it.</li>
<li><strong>Do you want to touch a very core component?</strong> In order to prevent
major regressions, pull requests that touch core components receive
extra scrutiny. Make sure you’ve discussed your changes with the team
before undertaking major changes.</li>
<li><strong>Want to add a new feature?</strong> If you want to add new features,
comment your intention on the related issue. Our team tries to
comment on and provide feedback to the community. It’s better to have
an open discussion with the team and the rest of the community prior
to building new features. This helps us stay aware of what you’re
working on and increases the chance that it’ll be merged.</li>
<li><strong>Did you touch unrelated code to the PR?</strong> To aid in code review,
please only include files in your pull request that are directly
related to your changes.</li>
</ul>
<p>Frequently asked questions</p>
<ul class="simple">
<li><strong>How can I contribute as a reviewer?</strong> There is lots of value if
community developer reproduce issues, try out new functionality, or
otherwise help us identify or troubleshoot issues. Commenting on
tasks or pull requests with your enviroment details is helpful and
appreciated.</li>
<li><strong>CI tests failed, what does it mean?</strong> Maybe you need to merge with
master or rebase with latest changes. Pushing your changes should
re-trigger CI tests. If the tests persist, you’ll want to trace
through the error messages and resolve the related issues.</li>
<li><strong>What are the most high risk changes?</strong> Anything that touches build
configuration is an risky area. Please avoid changing these unless
you’ve had a discussion with the team beforehand.</li>
<li><strong>Hey, a commit showed up on my branch, what’s up with that?</strong>
Sometimes another community member will provide a patch or fix to
your pull request or branch. This is often needed for getting CI tests
to pass.</li>
</ul>
</div>
<div class="section" id="on-documentation">
<h2>On Documentation<a class="headerlink" href="#on-documentation" title="Permalink to this headline">¶</a></h2>
<div class="section" id="python-docs">
<h3>Python Docs<a class="headerlink" href="#python-docs" title="Permalink to this headline">¶</a></h3>
<p>PyTorch documentation is generated from python source using
<a class="reference external" href="http://www.sphinx-doc.org/en/master/">Sphinx</a>. Generated HTML is
copied to the docs folder in the master branch of
<a class="reference external" href="https://github.com/pytorch/pytorch.github.io/tree/master/docs">pytorch.github.io</a>,
and is served via GitHub pages.</p>
<ul class="simple">
<li>Site: <a class="reference external" href="http://pytorch.org/docs">http://pytorch.org/docs</a></li>
<li>GitHub: <a class="reference external" href="http://github.com/pytorch/pytorch/docs">http://github.com/pytorch/pytorch/docs</a></li>
<li>Served from:
<a class="reference external" href="https://github.com/pytorch/pytorch.github.io/tree/master/docs">https://github.com/pytorch/pytorch.github.io/tree/master/doc</a></li>
</ul>
</div>
<div class="section" id="c-docs">
<h3>C++ Docs<a class="headerlink" href="#c-docs" title="Permalink to this headline">¶</a></h3>
<p>For C++ code we use Doxygen to generate the content files. The C++ docs
are built on a special server and the resulting files are copied to the
<a class="reference external" href="https://github.com/pytorch/cppdocs">https://github.com/pytorch/cppdocs</a> repo, and are served from GitHub
pages.</p>
<ul class="simple">
<li>Site: <a class="reference external" href="http://pytorch.org/cppdocs">http://pytorch.org/cppdocs</a></li>
<li>GitHub: <a class="reference external" href="https://github.com/pytorch/pytorch/tree/master/docs/cpp">https://github.com/pytorch/pytorch/tree/master/docs/cpp</a></li>
<li>Served from: <a class="reference external" href="https://github.com/pytorch/cppdocs">https://github.com/pytorch/cppdocs</a></li>
</ul>
</div>
</div>
<div class="section" id="tutorials">
<h2>Tutorials<a class="headerlink" href="#tutorials" title="Permalink to this headline">¶</a></h2>
<p>PyTorch tutorials are documents used to help understand using PyTorch to
accomplish specific tasks or to understand more holistic concepts.
Tutorials are built using
<a class="reference external" href="https://sphinx-gallery.readthedocs.io/en/latest/index.html">Sphinx-Gallery</a>
from executable python sources files, or from restructured-text (rst)
files.</p>
<ul class="simple">
<li>Site: <a class="reference external" href="http://pytorch.org/tutorials">http://pytorch.org/tutorials</a></li>
<li>GitHub: <a class="reference external" href="http://github.com/pytorch/tutorials">http://github.com/pytorch/tutorials</a></li>
</ul>
<div class="section" id="tutorials-build-overview">
<h3>Tutorials Build Overview<a class="headerlink" href="#tutorials-build-overview" title="Permalink to this headline">¶</a></h3>
<p>For tutorials, <a class="reference external" href="https://github.com/pytorch/tutorials/pulls">pull
requests</a> trigger a
rebuild the entire site using CircleCI to test the effects of the
change. This build is sharded into 9 worker builds and takes around 40
minutes total. At the same time, we do a Netlify build using <em>make
html-noplot</em>, which builds the site without rendering the notebook
output into pages for quick review.</p>
<p>After a PR is accepted, the site is rebuilt and deployed from CircleCI.</p>
</div>
<div class="section" id="contributing-a-new-tutorial">
<h3>Contributing a new Tutorial<a class="headerlink" href="#contributing-a-new-tutorial" title="Permalink to this headline">¶</a></h3>
<p><a class="reference external" href="https://github.com/pytorch/tutorials/#contributing">PyTorch.org Tutorial Contribution
Guide</a></p>
</div>
<div class="section" id="code-style">
<h3>Code Style<a class="headerlink" href="#code-style" title="Permalink to this headline">¶</a></h3>
<p><strong>Python style</strong></p>
<p><strong>C++ style</strong></p>
</div>
<div class="section" id="submitting-a-pull-request">
<h3>Submitting a Pull Request<a class="headerlink" href="#submitting-a-pull-request" title="Permalink to this headline">¶</a></h3>
<p>PyTorch development happens publicly on our Github repo.</p>
<p>To have your feature or fix added to PyTorch, please submit a Pull
Request.</p>
</div>
<div class="section" id="running-tests">
<h3>Running Tests<a class="headerlink" href="#running-tests" title="Permalink to this headline">¶</a></h3>
<p>Show examples for running all tests, just one individual…</p>
</div>
</div>
<div class="section" id="technical-process">
<h2>Technical Process<a class="headerlink" href="#technical-process" title="Permalink to this headline">¶</a></h2>
<div class="section" id="developing-pytorch">
<h3>Developing PyTorch<a class="headerlink" href="#developing-pytorch" title="Permalink to this headline">¶</a></h3>
<p>To develop PyTorch on your machine, here are some tips:</p>
<ol class="arabic simple">
<li>Uninstall all existing PyTorch installs:</li>
</ol>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">conda</span> <span class="n">uninstall</span> <span class="n">pytorch</span>
<span class="n">pip</span> <span class="n">uninstall</span> <span class="n">torch</span>
<span class="n">pip</span> <span class="n">uninstall</span> <span class="n">torch</span> <span class="c1"># run this command twice</span>
</pre></div>
</div>
<ol class="arabic simple" start="2">
<li>Clone a copy of PyTorch from source:</li>
</ol>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">git</span> <span class="n">clone</span> <span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="o">/</span><span class="n">pytorch</span><span class="o">/</span><span class="n">pytorch</span>
<span class="n">cd</span> <span class="n">pytorch</span>
</pre></div>
</div>
<ol class="arabic simple" start="3">
<li>Install PyTorch in <code class="docutils literal notranslate"><span class="pre">build</span> <span class="pre">develop</span></code> mode:</li>
</ol>
<p>A full set of instructions on installing PyTorch from source is here:
<a class="reference external" href="https://github.com/pytorch/pytorch#from-source">https://github.com/pytorch/pytorch#from-source</a></p>
<p>The change you have to make is to replace</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="n">setup</span><span class="o">.</span><span class="n">py</span> <span class="n">install</span>
</pre></div>
</div>
<p>with</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="n">setup</span><span class="o">.</span><span class="n">py</span> <span class="n">build</span> <span class="n">develop</span>
</pre></div>
</div>
<p>This is especially useful if you are only changing Python files.</p>
<p>This mode will symlink the Python files from the current local source
tree into the Python install.</p>
<p>Hence, if you modify a Python file, you do not need to reinstall PyTorch
again and again.</p>
<p>For example:</p>
<ul class="simple">
<li>Install local PyTorch in <code class="docutils literal notranslate"><span class="pre">build</span> <span class="pre">develop</span></code> mode</li>
<li>modify your Python file <code class="docutils literal notranslate"><span class="pre">torch/__init__.py</span></code> (for example)</li>
<li>test functionality</li>
<li>modify your Python file <code class="docutils literal notranslate"><span class="pre">torch/__init__.py</span></code></li>
<li>test functionality</li>
<li>modify your Python file <code class="docutils literal notranslate"><span class="pre">torch/__init__.py</span></code></li>
<li>test functionality</li>
</ul>
<p>You do not need to repeatedly install after modifying Python files.</p>
<p>In case you want to reinstall, make sure that you uninstall PyTorch
first by running <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">uninstall</span> <span class="pre">torch</span></code> and <code class="docutils literal notranslate"><span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">clean</span></code>.
Then you can install in <code class="docutils literal notranslate"><span class="pre">build</span> <span class="pre">develop</span></code> mode again.</p>
</div>
</div>
<div class="section" id="codebase-structure">
<h2>Codebase structure<a class="headerlink" href="#codebase-structure" title="Permalink to this headline">¶</a></h2>
<ul class="simple">
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/c10">c10</a> - Core
library files that work everywhere, both server and mobile. We are
slowly moving pieces from
<a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/core">ATen/core</a>
here. This library is intended only to contain essential
functionality, and appropriate to use in settings where binary size
matters. (But you’ll have a lot of missing functionality if you try
to use it directly.)</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten">aten</a> - C++
tensor library for PyTorch (no autograd support)<ul>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src">src</a><ul>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/TH">TH</a>
<a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/THC">THC</a>
<a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/THNN">THNN</a>
<a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/THCUNN">THCUNN</a>
- Legacy library code from the original Torch. Try not to add
things here; we’re slowly porting these to
<a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native">native</a>.<ul>
<li>generic - Contains actual implementations of operators,
parametrized over <code class="docutils literal notranslate"><span class="pre">scalar_t</span></code>. Files here get compiled N
times per supported scalar type in PyTorch.</li>
</ul>
</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen">ATen</a><ul>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/core">core</a>
- Core functionality of ATen. This is migrating to top-level
c10 folder.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native">native</a>
- Modern implementations of operators. If you want to write
a new operator, here is where it should go. Most CPU
operators go in the top level directory, except for
operators which need to be compiled specially; see cpu
below.<ul>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/cpu">cpu</a>
- Not actually CPU implementations of operators, but
specifically implementations which are compiled with
processor-specific instructions, like AVX. See the
<a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/cpu/README.md">README</a>
for more details.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/cuda">cuda</a>
- CUDA implementations of operators.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/sparse">sparse</a>
- CPU and CUDA implementations of COO sparse tensor
operations</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/mkl">mkl</a>
<a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/mkldnn">mkldnn</a>
<a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/miopen">miopen</a>
<a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/cudnn">cudnn</a><ul>
<li>implementations of operators which simply bind to some
backend library.</li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/torch">torch</a> -
The actual PyTorch library. Everything that is not in
<a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/torch/csrc">csrc</a>
is a Python module, following the PyTorch Python frontend module
structure.<ul>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/torch/csrc">csrc</a>
- C++ files composing the PyTorch library. Files in this directory
tree are a mix of Python binding code, and C++ heavy lifting.
Consult <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> for the canonical list of Python binding
files; conventionally, they are often prefixed with <code class="docutils literal notranslate"><span class="pre">python_</span></code>.<ul>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/torch/csrc/jit">jit</a>
- Compiler and frontend for TorchScript JIT frontend.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/torch/csrc/autograd">autograd</a>
- Implementation of reverse-mode automatic differentiation.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/torch/csrc/api">api</a>
- The PyTorch C++ frontend.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/torch/csrc/distributed">distributed</a>
- Distributed training support for PyTorch.</li>
</ul>
</li>
</ul>
</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/tools">tools</a> -
Code generation scripts for the PyTorch library. See
<a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/tools/README.md">README</a>
of this directory for more details.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/tests">test</a> -
Python unit tests for PyTorch Python frontend.<ul>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/test/test_torch.py">test_torch.py</a>
- Basic tests for PyTorch functionality.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/test/test_autograd.py">test_autograd.py</a>
- Tests for non-NN automatic differentiation support.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/test/test_nn.py">test_nn.py</a>
- Tests for NN operators and their automatic differentiation.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/test/test_jit.py">test_jit.py</a>
- Tests for the JIT compiler and TorchScript.</li>
<li>…</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/test/cpp">cpp</a>
- C++ unit tests for PyTorch C++ frontend.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/test/expect">expect</a>
- Automatically generated “expect” files which are used to compare
against expected output.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/test/onnx">onnx</a>
- Tests for ONNX export functionality, using both PyTorch and
Caffe2.</li>
</ul>
</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/caffe2">caffe2</a> -
The Caffe2 library.<ul>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/caffe2/core">core</a>
- Core files of Caffe2, e.g., tensor, workspace, blobs, etc.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/caffe2/operators">operators</a>
- Operators of Caffe2.</li>
<li><a class="reference external" href="https://github.com/pytorch/pytorch/blob/master/caffe2/python">python</a>
- Python bindings to Caffe2.</li>
<li>…</li>
</ul>
</li>
</ul>
</div>
<div class="section" id="unit-testing">
<h2>Unit Testing<a class="headerlink" href="#unit-testing" title="Permalink to this headline">¶</a></h2>
<p>PyTorch’s testing is located under <code class="docutils literal notranslate"><span class="pre">test/</span></code>. Run the entire test suite
with</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="n">test</span><span class="o">/</span><span class="n">run_test</span><span class="o">.</span><span class="n">py</span>
</pre></div>
</div>
<p>or run individual test files, like <code class="docutils literal notranslate"><span class="pre">python</span> <span class="pre">test/test_nn.py</span></code>, for
individual test suites.</p>
<div class="section" id="better-local-unit-tests-with-pytest">
<h3>Better local unit tests with pytest<a class="headerlink" href="#better-local-unit-tests-with-pytest" title="Permalink to this headline">¶</a></h3>
<p>We don’t officially support <code class="docutils literal notranslate"><span class="pre">pytest</span></code>, but it works well with our
<code class="docutils literal notranslate"><span class="pre">unittest</span></code> tests and offers a number of useful features for local
developing. Install it via <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">pytest</span></code>.</p>
<p>If you want to just run tests that contain a specific substring, you can
use the <code class="docutils literal notranslate"><span class="pre">-k</span></code> flag:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">pytest</span> <span class="n">test</span><span class="o">/</span><span class="n">test_nn</span><span class="o">.</span><span class="n">py</span> <span class="o">-</span><span class="n">k</span> <span class="n">Loss</span> <span class="o">-</span><span class="n">v</span>
</pre></div>
</div>
<p>The above is an example of testing a change to Loss functions: this
command runs tests such as <code class="docutils literal notranslate"><span class="pre">TestNN.test_BCELoss</span></code>and
<code class="docutils literal notranslate"><span class="pre">TestNN.test_MSELoss</span></code> and can be useful to save keystrokes.</p>
</div>
</div>
<div class="section" id="writing-documentation">
<h2>Writing documentation<a class="headerlink" href="#writing-documentation" title="Permalink to this headline">¶</a></h2>
<p>PyTorch uses <a class="reference external" href="http://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html">Google
style</a>
for formatting docstrings. Length of line inside docstrings block must
be limited to 80 characters to fit into Jupyter documentation popups.</p>
<p>For C++ documentation (<a class="reference external" href="https://pytorch.org/cppdocs">https://pytorch.org/cppdocs</a>), we use
<a class="reference external" href="http://www.doxygen.nl/">Doxygen</a> and then convert it to
<a class="reference external" href="http://www.sphinx-doc.org/">Sphinx</a> via
<a class="reference external" href="https://github.com/michaeljones/breathe">Breathe</a>
and<a class="reference external" href="https://github.com/svenevs/exhale">Exhale</a>. Check the <a class="reference external" href="http://www.stack.nl/~dimitri/doxygen/manual/index.html">Doxygen
reference</a>
for more information on the documentation syntax. To build the
documentation locally, <code class="docutils literal notranslate"><span class="pre">cd</span></code> into <code class="docutils literal notranslate"><span class="pre">docs/cpp</span></code> and then <code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">html</span></code>.</p>
<p>We run Doxygen in CI (Travis) to verify that you do not use invalid
Doxygen commands. To run this check locally, run <code class="docutils literal notranslate"><span class="pre">./check-doxygen.sh</span></code>
from inside <code class="docutils literal notranslate"><span class="pre">docs/cpp</span></code>.</p>
</div>
<div class="section" id="managing-multiple-build-trees">
<h2>Managing multiple build trees<a class="headerlink" href="#managing-multiple-build-trees" title="Permalink to this headline">¶</a></h2>
<p>One downside to using <code class="docutils literal notranslate"><span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">develop</span></code> is that your
development version of PyTorch will be installed globally on your
account (e.g., if you run <code class="docutils literal notranslate"><span class="pre">import</span> <span class="pre">torch</span></code> anywhere else, the
development version will be used.</p>
<p>If you want to manage multiple builds of PyTorch, you can make use of
<a class="reference external" href="https://conda.io/docs/using/envs.html">conda environments</a> to
maintain separate Python package environments, each of which can be tied
to a specific build of PyTorch. To set one up:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">conda</span> <span class="n">create</span> <span class="o">-</span><span class="n">n</span> <span class="n">pytorch</span><span class="o">-</span><span class="n">myfeaturesource</span> <span class="n">activate</span> <span class="n">pytorch</span><span class="o">-</span><span class="n">myfeature</span><span class="c1"># if you run python now, torch will NOT be installed</span>
<span class="n">python</span> <span class="n">setup</span><span class="o">.</span><span class="n">py</span> <span class="n">build</span> <span class="n">develop</span>
</pre></div>
</div>
</div>
<div class="section" id="c-development-tips">
<h2>C++ Development tips<a class="headerlink" href="#c-development-tips" title="Permalink to this headline">¶</a></h2>
<p>If you are working on the C++ code, there are a few important things
that you will want to keep in mind:</p>
<ol class="arabic simple">
<li>How to rebuild only the code you are working on.</li>
<li>How to make rebuilds in the absence of changes go faster.</li>
</ol>
<div class="section" id="build-only-what-you-need">
<h3>Build only what you need.<a class="headerlink" href="#build-only-what-you-need" title="Permalink to this headline">¶</a></h3>
<p><code class="docutils literal notranslate"><span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">build</span></code> will build everything, but since our build
system is not very optimized for incremental rebuilds, this will
actually be very slow. Far better is to only request rebuilds of the
parts of the project you are working on:</p>
<ul class="simple">
<li>Working on the Python bindings? Run <code class="docutils literal notranslate"><span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">develop</span></code> to
rebuild (NB: no <code class="docutils literal notranslate"><span class="pre">build</span></code> here!)</li>
<li>Working on <code class="docutils literal notranslate"><span class="pre">torch/csrc</span></code> or <code class="docutils literal notranslate"><span class="pre">aten</span></code>? Run
<code class="docutils literal notranslate"><span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">rebuild_libtorch</span></code> to rebuild and avoid having to
rebuild other dependent libraries we depend on.</li>
<li>Working on one of the other dependent libraries? The other valid
targets are listed in <code class="docutils literal notranslate"><span class="pre">dep_libs</span></code> in <code class="docutils literal notranslate"><span class="pre">setup.py</span></code>. prepend
<code class="docutils literal notranslate"><span class="pre">build_</span></code> to get a target, and run as e.g.
<code class="docutils literal notranslate"><span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">build_gloo</span></code>.</li>
<li>Working on a test binary? Run
<code class="docutils literal notranslate"><span class="pre">(cd</span> <span class="pre">build</span> <span class="pre">&&</span> <span class="pre">ninja</span> <span class="pre">bin/test_binary_name)</span></code> to rebuild only that
test binary (without rerunning cmake). (Replace <code class="docutils literal notranslate"><span class="pre">ninja</span></code> with
<code class="docutils literal notranslate"><span class="pre">make</span></code> if you don’t have ninja installed).</li>
</ul>
<p>On the initial build, you can also speed things up with the environment
variables <code class="docutils literal notranslate"><span class="pre">DEBUG</span></code> and <code class="docutils literal notranslate"><span class="pre">NO_CUDA</span></code>.</p>
<ul class="simple">
<li><code class="docutils literal notranslate"><span class="pre">DEBUG=1</span></code> will enable debug builds (-g -O0)</li>
<li><code class="docutils literal notranslate"><span class="pre">REL_WITH_DEB_INFO=1</span></code> will enable debug symbols with optimizations
(-g -O3)</li>
<li><code class="docutils literal notranslate"><span class="pre">NO_CUDA=1</span></code> will disable compiling CUDA (in case you are developing
on something not CUDA related), to save compile time.</li>
</ul>
<p>For example:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">NO_CUDA</span><span class="o">=</span><span class="mi">1</span> <span class="n">DEBUG</span><span class="o">=</span><span class="mi">1</span> <span class="n">python</span> <span class="n">setup</span><span class="o">.</span><span class="n">py</span> <span class="n">build</span> <span class="n">develop</span>
</pre></div>
</div>
<p>Make sure you continue to pass these flags on subsequent builds.</p>
</div>
<div class="section" id="code-completion-and-ide-support">
<h3>Code completion and IDE support<a class="headerlink" href="#code-completion-and-ide-support" title="Permalink to this headline">¶</a></h3>
<p>When using <code class="docutils literal notranslate"><span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">develop</span></code>, PyTorch will generate a
<code class="docutils literal notranslate"><span class="pre">compile_commands.json</span></code> file that can be used by many editors to
provide command completion and error highlighting for PyTorch’s C++
code. You need to <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">ninja</span></code> to generate accurate information
for the code in <code class="docutils literal notranslate"><span class="pre">torch/csrc</span></code>. More information at:</p>
<ul class="simple">
<li><a class="reference external" href="https://sarcasm.github.io/notes/dev/compilation-database.html">https://sarcasm.github.io/notes/dev/compilation-database.html</a></li>
</ul>
</div>
<div class="section" id="make-no-op-build-fast">
<h3>Make no-op build fast.<a class="headerlink" href="#make-no-op-build-fast" title="Permalink to this headline">¶</a></h3>
</div>
<div class="section" id="use-ninja">
<h3>Use Ninja<a class="headerlink" href="#use-ninja" title="Permalink to this headline">¶</a></h3>
<p>Python <code class="docutils literal notranslate"><span class="pre">setuptools</span></code> is pretty dumb, and always rebuilds every C file
in a project. If you install the ninja build system with
<code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">ninja</span></code>, then PyTorch will use it to track dependencies
correctly. If PyTorch was already built, you will need to run
<code class="docutils literal notranslate"><span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">clean</span></code> once after installing ninja for builds to
succeed.</p>
</div>
<div class="section" id="use-ccache">
<h3>Use CCache<a class="headerlink" href="#use-ccache" title="Permalink to this headline">¶</a></h3>
<p>Even when dependencies are tracked with file modification, there are
many situations where files get rebuilt when a previous compilation was
exactly the same.</p>
<p>Using ccache in a situation like this is a real time-saver. However, by
default, ccache does not properly support CUDA stuff, so here are the
instructions for installing a custom ccache fork that has CUDA support:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span># install and export ccacheif ! ls ~/ccache/bin/ccachethen
sudo apt-get update
sudo apt-get install -y automake autoconf
sudo apt-get install -y asciidoc
mkdir -p ~/ccache
pushd /tmp
rm -rf ccache
git clone https://github.com/colesbury/ccache -b ccbin
pushd ccache
./autogen.sh
./configure
make install prefix=~/ccache
popdpopd
mkdir -p ~/ccache/lib
mkdir -p ~/ccache/cuda
ln -s ~/ccache/bin/ccache ~/ccache/lib/cc
ln -s ~/ccache/bin/ccache ~/ccache/lib/c++
ln -s ~/ccache/bin/ccache ~/ccache/lib/gcc
ln -s ~/ccache/bin/ccache ~/ccache/lib/g++
ln -s ~/ccache/bin/ccache ~/ccache/cuda/nvcc
~/ccache/bin/ccache -M 25Gifiexport PATH=~/ccache/lib:$PATHexport CUDA_NVCC_EXECUTABLE=~/ccache/cuda/nvcc
</pre></div>
</div>
</div>
</div>
<div class="section" id="cuda-development-tips">
<h2>CUDA Development tips<a class="headerlink" href="#cuda-development-tips" title="Permalink to this headline">¶</a></h2>
<p>If you are working on the CUDA code, here are some useful CUDA debugging
tips:</p>
<ol class="arabic simple">
<li><code class="docutils literal notranslate"><span class="pre">CUDA_DEVICE_DEBUG=1</span></code> will enable CUDA device function debug
symbols (<code class="docutils literal notranslate"><span class="pre">-g</span> <span class="pre">-G</span></code>). This will be particularly helpful in debugging
device code. However, it will slow down the build process for about
50% (compared to only <code class="docutils literal notranslate"><span class="pre">DEBUG=1</span></code>), so use wisely.</li>
<li><code class="docutils literal notranslate"><span class="pre">cuda-gdb</span></code> and <code class="docutils literal notranslate"><span class="pre">cuda-memcheck</span></code> are your best CUDA debugging
friends. Unlike<code class="docutils literal notranslate"><span class="pre">gdb</span></code>, <code class="docutils literal notranslate"><span class="pre">cuda-gdb</span></code> can display actual values in a
CUDA tensor (rather than all zeros).</li>
</ol>
<p>Hope this helps, and thanks for considering to contribute.</p>
</div>
<div class="section" id="windows-development-tips">
<h2>Windows development tips<a class="headerlink" href="#windows-development-tips" title="Permalink to this headline">¶</a></h2>
<p>Occasionally, you will write a patch which works on Linux, but fails CI
on Windows. There are a few aspects in which MSVC (the Windows compiler
toolchain we use) is stricter than Linux, which are worth keeping in
mind when fixing these problems.</p>
<ol class="arabic simple">
<li>Symbols are NOT exported by default on Windows; instead, you have to
explicitly mark a symbol as exported/imported in a header file with
<code class="docutils literal notranslate"><span class="pre">__declspec(dllexport)</span></code> / <code class="docutils literal notranslate"><span class="pre">__declspec(dllimport)</span></code>. We have
codified this pattern into a set of macros which follow the
convention <code class="docutils literal notranslate"><span class="pre">*_API</span></code>, e.g., <code class="docutils literal notranslate"><span class="pre">CAFFE2_API</span></code> inside Caffe2 and ATen.
(Every separate shared library needs a unique macro name, because
symbol visibility is on a per shared library basis. See
c10/macros/Macros.h for more details.) The upshot is if you see an
“unresolved external” error in your Windows build, this is probably
because you forgot to mark a function with <code class="docutils literal notranslate"><span class="pre">*_API</span></code>. However, there
is one important counterexample to this principle: if you want a
<em>templated</em> function to be instantiated at the call site, do NOT mark
it with <code class="docutils literal notranslate"><span class="pre">*_API</span></code> (if you do mark it, you’ll have to explicitly
instantiate all of the specializations used by the call sites.)</li>
<li>If you link against a library, this does not make its dependencies
transitively visible. You must explicitly specify a link dependency
against every library whose symbols you use. (This is different from
Linux where in most environments, transitive dependencies can be used
to fulfill unresolved symbols.)</li>
<li>If you have a Windows box (we have a few on EC2 which you can request
access to) and you want to run the build, the easiest way is to just
run <code class="docutils literal notranslate"><span class="pre">.jenkins/pytorch/win-build.sh</span></code>. If you need to rebuild, run
<code class="docutils literal notranslate"><span class="pre">REBUILD=1</span> <span class="pre">.jenkins/pytorch/win-build.sh</span></code> (this will avoid blowing
away your Conda environment.)</li>
</ol>
<p>Even if you don’t know anything about MSVC, you can use cmake to build
simple programs on Windows; this can be helpful if you want to learn
more about some peculiar linking behavior by reproducing it on a small