[dev/arcade-migration] Improve dev scenarios (dotnet/core-setup#7233)

* Add '/p:Subset=' support for dev scenarios

* Fix readme table generation

* Improve VS build: update helper projects, doc

* Document Subsets better; add polish and safety

* Update windows-instructions.md, add Subsets hint


Commit migrated from dotnet/core-setup@c9c64e7

docs/installer/building/Running-Tests-In-VisualStudio-IDE.md

@@ -1,19 +1,22 @@
 # Running unit tests within Visual Studio
 
-Sometimes it is convenient to run individual unit tests within the Visual Studio IDE
-
-There are several environment variables which must be set to get this to work correctly.  To make things easier there is a convenience script `visual-studio-devenv.cmd` provided.  This script will set the necessary environment variables.
+Sometimes it is convenient to run individual unit tests within the Visual Studio
+IDE. First, build the repo from the command line to create artifacts and set up
+the test environment. Then, use VS Test Explorer to run and debug tests.
 
 ## Steps
 
-1. `build.cmd`
-2. `visual-studio-devenv.cmd`
-3. Open the test explorer window within the Visual Studio IDE
+1. `build.cmd -test`
+2. Open the solution file in the root of the repo.
+3. Open the test explorer window within the Visual Studio IDE.
 4. Select tests and run and/or debug.
 
 ## Limitations
 
-* The script is not yet designed to be a robust solution.  As test configurations change this file could get out of date.  The required configuration can be discovered by carefully examining the logs generated when running the tests with `build.cmd -MsBuildLogging=/bl`.  The script can then be updated.
-* The script is hardcoded to use the x64 debug build
-
-
+* The managed projects load and build, but native and setup projects are not
+  present in the solution and there's no way to trigger a build from inside VS.
+* Rebuilding the native assets alone won't make them used during tests. The
+  tests rely on the setup projects to assemble the native bits into a usable
+  form, and they have to be rebuilt.
+  * With a deep enough understanding of the test layout, you can work around
+    this by copying native build outputs directly into the test layout.

docs/installer/building/windows-instructions.md

@@ -34,9 +34,10 @@ If using Visual Studio 2019, then at least CMake 3.14 is required.
 
 ## Git
 
-For actual user operations, it is often more convinient to use the GIT features built into Visual Studio 2015.
-However the Core-Setup and the tests use the GIT command line utilities directly so you need to install them
-for these to work properly.   You can get it from 
+For actual user operations, it is often convenient to use the Git features built
+into your editor or IDE. However, Core-Setup and the tests use the Git command
+line utilities directly, so you need to set them up for the build to work
+properly. You can get Git from:
 
 - Install [Git For Windows](https://git-for-windows.github.io/)
 - Add its location (e.g. C:\Program Files\Git\cmd) to the PATH environment variable.  
@@ -50,11 +51,6 @@ Typically this is %SYSTEMROOT%\System32\WindowsPowerShell\v1.0\.
 Powershell version must be 3.0 or higher. This should be the case for Windows 8 and later builds.
 - Windows 7 SP1 can install Powershell version 4 [here](https://www.microsoft.com/en-us/download/details.aspx?id=40855).
 
-## DotNet Core SDK
-While not strictly needed to build or tests the .NET Core repository, having the .NET Core SDK installed lets 
-you use the dotnet.exe command to run .NET Core applications in the 'normal' way.  Visual Studio 2015 (update 3) should have
-installed the .NET Core SDK, but in case it did not you can get it from the [Installing the .Net Core SDK](https://www.microsoft.com/net/core) page.  
-
 ## Adding to the default PATH variable
 
 The commands above need to be on your command lookup path.   Some installers will automatically add them to 
@@ -67,10 +63,16 @@ You can of course add a directory to the PATH environment variable with the synt
 However the change above will only last until the command windows closes.   You can make your change to
 the PATH variable persistent by going to  Control Panel -> System And Security -> System -> Advanced system settings -> Environment Variables, 
 and select the 'Path' variable in the 'System variables' (if you want to change it for all users) or 'User variables' (if you only want
-to change it for the currnet user).  Simply edit the PATH variable's value and add the directory (with a semicolon separator).
+to change it for the current user).  Simply edit the PATH variable's value and add the directory (with a semicolon separator).
 
 -------------------------------------
 # Building
 
-Once all the necessary tools are in place, building is trivial.  Simply run build build.cmd script that lives at
-the base of the repository.   
+Once all the necessary tools are in place, building is trivial.  Simply run the
+`build.cmd` script that lives at the base of the repository.
+
+If you want to build a subset of the build because `build.cmd` takes too long,
+try using the `Subset` property. For example, adding `/p:Subset=CoreHost` to
+your build command makes it only build the native project. Read the
+documentation in [Subsets.props](/Subsets.props) and try `/p:Subset=help` for
+more info.

src/installer/Subsets.props

@@ -0,0 +1,129 @@
+<Project>
+
+  <!--
+    This file defines the list of projects to build and divides them into subsets. In ordinary
+    situations, you should perform a full build by running 'build.cmd' or './build.sh'. This ensures
+    the projects are sequenced correctly so the outputs and test results are what you would expect.
+
+    If you know you only want to run a subset of the build, however, use the Subset property.
+
+    Syntax:
+
+      (build.cmd/sh) /p:Subset=<desired subset name 1>(-<desired subset name N>)*
+
+      - For a description of each subset, use '/p:Subset=help'.
+      - Subset names are case insensitive.
+      - 'Subset' is case insensitive. (That is, '/p:subset' works.)
+      - Order doesn't affect the result.
+
+    Examples:
+
+      ./build.sh /p:Subset=CoreHost
+
+        This builds only the .NET Core Host.
+
+      ./build.sh /p:Subset=CoreHost-Managed
+
+        This builds the CoreHost and also the Managed (e.g. Microsoft.Extensions.DependencyModel)
+        projects. A '-' is the delimiter between multiple subsets to build.
+
+      ./build.sh -test /p:Subset=Test
+
+        This builds and executes the test projects. (The '-test' argument is an Arcade SDK argument
+        that indicates tests should be run. Otherwise, they'll only be built.)
+
+    Quirks:
+
+      This command looks useful, but doesn't work as expected:
+      
+        ./build.sh -test /p:Subset=CoreHost-Test    # (Doesn't work!)
+
+      Intuitively, this should build the host, build the tests, then run the tests on the freshly
+      built host. What actually happens is the tests run on a previously built host. This is because
+      the depproj, pkgproj, and installer subsets process the host artifacts, and those didn't
+      rebuild because those subsets were disabled.
+
+      You can get around this limitation by running the corehost subset, manually copying host
+      artifacts to the test layout, then running the test subset.
+  -->
+
+  <PropertyGroup>
+    <SubsetToLower>$(Subset.ToLowerInvariant())</SubsetToLower>
+  </PropertyGroup>
+
+  <ItemGroup>
+    <SubsetName
+      Include="CoreHost"
+      Description="The .NET Core Host projects. This includes all the native code in Core-Setup." />
+  </ItemGroup>
+  <ItemGroup Condition="'$(SubsetToLower)' == '' or $(SubsetToLower.Contains('corehost'))">
+    <CorehostProjectToBuild Include="$(RepoRoot)src\corehost\build.proj" SignPhase="Binaries" />
+    <ProjectToBuild Include="@(CorehostProjectToBuild)" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <SubsetName
+      Include="Managed"
+      Description="The managed .NET projects. This includes PlatformAbstractions, DependencyModel, and HostModel." />
+  </ItemGroup>
+  <ItemGroup Condition="'$(SubsetToLower)' == '' or $(SubsetToLower.Contains('managed'))">
+    <ManagedProjectToBuild Include="$(RepoRoot)src\managed\**\*.csproj" SignPhase="Binaries" />
+    <ManagedProjectToBuild Include="$(RepoRoot)src\pkg\packaging\pack-managed.proj" />
+    <ProjectToBuild Include="@(ManagedProjectToBuild)" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <SubsetName
+      Include="depproj"
+      Description="The dependency projects. These gather shared framework files and run crossgen on them to turn them into ready-to-run (R2R) assemblies for the current platform." />
+  </ItemGroup>
+  <ItemGroup Condition="'$(SubsetToLower)' == '' or $(SubsetToLower.Contains('depproj'))">
+    <DepprojProjectToBuild Include="$(RepoRoot)src\pkg\projects\**\*.depproj" SignPhase="Binaries" />
+    <ProjectToBuild Include="@(DepprojProjectToBuild)" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <SubsetName
+      Include="pkgproj"
+      Description="The packaging projects. These produce NETCoreApp and WindowsDesktop assets: NuGet packages, installers, zips, and Linux packages." />
+  </ItemGroup>
+  <ItemGroup Condition="'$(SubsetToLower)' == '' or $(SubsetToLower.Contains('pkgproj'))">
+    <PkgprojProjectToBuild Include="$(RepoRoot)src\pkg\projects\**\*.pkgproj" SignPhase="MsiFiles" />
+    <ProjectToBuild Include="@(PkgprojProjectToBuild)" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <SubsetName
+      Include="Installer"
+      Description="Generates additional installers. This produces the shared frameworks and their installers." />
+  </ItemGroup>
+  <ItemGroup Condition="'$(SubsetToLower)' == '' or $(SubsetToLower.Contains('installer'))">
+    <InstallerProjectToBuild Include="$(RepoRoot)src\pkg\packaging\installers.proj" />
+    <ProjectToBuild Include="@(InstallerProjectToBuild)" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <SubsetName
+      Include="Test"
+      Description="The test projects. Note that building this doesn't execute tests: you must also pass the '-test' argument." />
+  </ItemGroup>
+  <ItemGroup Condition="'$(SubsetToLower)' == '' or $(SubsetToLower.Contains('test'))">
+    <TestProjectToBuild Include="$(RepoRoot)src\test\BundleTests\AppHost.Bundle.Tests\AppHost.Bundle.Tests.csproj" />
+    <TestProjectToBuild Include="$(RepoRoot)src\test\BundleTests\Microsoft.NET.HostModel.Bundle.Tests\Microsoft.NET.HostModel.Bundle.Tests.csproj" />
+    <TestProjectToBuild Include="$(RepoRoot)src\test\HostActivation.Tests\HostActivation.Tests.csproj" />
+    <TestProjectToBuild Include="$(RepoRoot)src\test\Microsoft.DotNet.CoreSetup.Packaging.Tests\Microsoft.DotNet.CoreSetup.Packaging.Tests.csproj" />
+    <TestProjectToBuild Include="$(RepoRoot)src\test\Microsoft.Extensions.DependencyModel.Tests\Microsoft.Extensions.DependencyModel.Tests.csproj" />
+    <ProjectToBuild Include="@(TestProjectToBuild)" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <SubsetName
+      Include="RegenerateReadmeTable"
+      Description="Regenerates the table of asset links in the README.md file."
+      OnDemand="true"/>
+  </ItemGroup>
+  <ItemGroup Condition="$(SubsetToLower.Contains('regeneratereadmetable'))">
+    <ProjectToBuild Include="$(RepoRoot)tools-local\regenerate-readme-table.proj" />
+  </ItemGroup>
+
+</Project>

src/installer/pkg/projects/Directory.Build.props

@@ -43,7 +43,7 @@
 
   <!-- Most packages need the host to be built first. -->
   <ItemGroup Condition="'$(DisableOrderDependencies)' != 'true'">
-    <OrderProjectReference Include="$(SourceDir)corehost\build.proj" />
+    <OrderProjectReference Include="@(CorehostProjectToBuild)" />
     <!-- Pkgproj requires its depproj crossgen outputs to be signed.-->
     <OrderProjectReference
       Condition="'$(MSBuildProjectExtension)' == '.pkgproj'"

src/installer/signing/Directory.Build.props

@@ -2,9 +2,6 @@
 
   <Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory).., Directory.Build.props))\Directory.Build.props" />
 
-  <!-- Some signing stages check Build.props to find which projects to depend on. -->
-  <Import Project="$(RepositoryEngineeringDir)Build.props" />
-
   <PropertyGroup>
     <TargetFramework>$(NETCoreAppFramework)</TargetFramework>
 

src/installer/test/BundleTests/Helpers/BundleHelper.csproj

@@ -2,7 +2,7 @@
 
   <PropertyGroup>
     <Description>Bundle Test Helpers</Description>
-    <TargetFramework>netcoreapp2.0</TargetFramework>
+    <TargetFramework>$(NETCoreAppFramework)</TargetFramework>
     <AssemblyName>BundleHelper</AssemblyName>
     <GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
   </PropertyGroup>

src/installer/test/Microsoft.DotNet.CoreSetup.Packaging.Tests/Microsoft.DotNet.CoreSetup.Packaging.Tests.csproj

@@ -16,7 +16,7 @@
 
   <ItemGroup>
     <ProjectReference Include="..\TestUtils\TestUtils.csproj" />
-    <OrderProjectReference Include="$(SourceDir)pkg\projects\**\*.pkgproj" />
+    <OrderProjectReference Include="@(PkgprojProjectToBuild)" />
   </ItemGroup>
 
   <!--

src/installer/test/PrepareTestAssets/PrepareTestAssets.proj

@@ -4,8 +4,16 @@
           DependsOnTargets="
             GetProductVersions;
             DetermineTestOutputDirectory">
-    <!-- Ensure installers (and therefore shared framework projects) are built first. -->
-    <MSBuild Projects="$(SourceDir)pkg\packaging\installers.proj" Targets="Build" />
+    <!--
+      Ensure installers (and therefore shared framework projects) are built first. Include used
+      transitive dependenices here in case Subset is defined.
+    -->
+    <MSBuild
+      Projects="
+        @(ManagedProjectToBuild);
+        @(PkgprojProjectToBuild);
+        @(InstallerProjectToBuild)"
+      Targets="Build" />
 
     <PropertyGroup>
       <TempFolderRoot>$(IntermediateOutputPath)temp\</TempFolderRoot>

src/installer/test/TestUtils/TestUtils.csproj

@@ -1,11 +1,10 @@
 <Project Sdk="Microsoft.NET.Sdk">
 
   <PropertyGroup>
-    <TargetFramework>netcoreapp1.0</TargetFramework>
+    <TargetFramework>$(NETCoreAppFramework)</TargetFramework>
     <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
     <AssemblyName>TestUtils</AssemblyName>
     <PackageId>TestUtils</PackageId>
-    <RuntimeFrameworkVersion>1.0.4</RuntimeFrameworkVersion>
   </PropertyGroup>
 
   <ItemGroup>

tools-local/regenerate-readme-table.proj

@@ -1,8 +1,9 @@
-<Project Sdk="Microsoft.NET.Sdk" DefaultTargets="RegenerateReadmeTable">
+<Project>
+  <Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory).., Directory.Build.props))\Directory.Build.props" />
 
   <UsingTask TaskName="RegenerateReadmeTable" AssemblyFile="$(LocalBuildToolsTaskFile)" />
 
-  <Target Name="RegenerateReadmeTable">
+  <Target Name="Build">
     <!--
       Column and row configuration for the readme table. The abbreviations match up with
       reference-style Markdown links in the readme file to figure out what to put in the cells.
@@ -41,6 +42,9 @@
       ReadmeFile="$(RepoRoot)README.md"
       Branches="@(Branches)"
       Platforms="@(Platforms)" />
+
+    <Message Text="$(MSBuildProjectName) -> $(RepoRoot)README.md" Importance="High" />
   </Target>
 
+  <Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory).., Directory.Build.targets))\Directory.Build.targets" />
 </Project>