Skip to content

Commit

Permalink
[vcpkg] Rename port_versions to versions (microsoft#15784)
Browse files Browse the repository at this point in the history
  • Loading branch information
strega-nil authored Jan 21, 2021
1 parent bedeaa6 commit 68a7495
Show file tree
Hide file tree
Showing 1,613 changed files with 73 additions and 74 deletions.
10 changes: 5 additions & 5 deletions docs/maintainers/maintainer-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,16 +204,16 @@ For Example:

See our [manifest files document](manifest-files.md#port-version) for a full explanation of our conventions.

### Update the version files in `port_versions/` of any modified ports
### Update the version files in `versions/` of any modified ports

Vcpkg uses a set of metadata files to power its versioning feature.
These files are located in the following locations:
* `${VCPKG_ROOT}/port_versions/baseline.json`, (this file is common to all ports) and
* `${VCPKG_ROOT}/port_versions/${first-letter-of-portname}-/${portname}.json` (one per port).
* `${VCPKG_ROOT}/versions/baseline.json`, (this file is common to all ports) and
* `${VCPKG_ROOT}/versions/${first-letter-of-portname}-/${portname}.json` (one per port).

For example, for `zlib` the relevant files are:
* `${VCPKG_ROOT}/port_versions/baseline.json`
* `${VCPKG_ROOT}/port_versions/z-/zlib.json`
* `${VCPKG_ROOT}/versions/baseline.json`
* `${VCPKG_ROOT}/versions/z-/zlib.json`

We expect that each time you update a port, you also update its version files.

Expand Down
10 changes: 5 additions & 5 deletions docs/specifications/versioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ Baselines are used to set lower boundaries on package versions. A baseline effec

But what is a baseline?

In the main registry, the baseline is a file located in `${VCPKG_ROOT}/port_versions/baseline.json`. This file contains a version declaration for each package in vcpkg. The format of this file is the following:
In the main registry, the baseline is a file located in `${VCPKG_ROOT}/versions/baseline.json`. This file contains a version declaration for each package in vcpkg. The format of this file is the following:

```json
{
Expand Down Expand Up @@ -319,9 +319,9 @@ Although the concept of package versions has always been present in vcpkg, the c

With the introduction of versioning constraints, it is now possible that a package depends on a port version that does not match the one available locally. This raises a problem as vcpkg needs to know how to acquire the port files for the requested version.

To solve this problem, a new set of metadata needs to be introduced. This specification proposes a that a new “port_versions” folder is added as part of a registry. In the main vcpkg registry, this means a new root level port_versions directory.
To solve this problem, a new set of metadata needs to be introduced. This specification proposes a that a new "versions" folder is added as part of a registry. In the main vcpkg registry, this means a new root level versions directory.

The port_versions directory, from here on referred as the versions database, will contain JSON files for each one of the ports available in the registry. Each file will list all the versions available for a package and contain a Git tree-ish object that vcpkg can check out to obtain that version’s portfiles.
The versions directory, from here on referred as the versions database, will contain JSON files for each one of the ports available in the registry. Each file will list all the versions available for a package and contain a Git tree-ish object that vcpkg can check out to obtain that version’s portfiles.

As part of the versioning implementation, a generator for these database files will be implemented. The generator will extract from our repository’s Git history, all the versions of each port that had been available at any moment in time and compile them into these database files.

Expand Down Expand Up @@ -349,5 +349,5 @@ Example: generated `zlib.json`
}
```

For each port, its corresponding versions file should be located in `port_versions/{first letter of port name}-/{port name}.json`. For example, zlib’s version file will be located in `port_versions/z-/zlib.json`.
Aside from port version files, the current baseline file is located in `port_versions/baseline.json`.
For each port, its corresponding versions file should be located in `versions/{first letter of port name}-/{port name}.json`. For example, zlib’s version file will be located in `versions/z-/zlib.json`.
Aside from port version files, the current baseline file is located in `versions/baseline.json`.
16 changes: 8 additions & 8 deletions scripts/azure-pipelines/end-to-end-tests-dir/registries.ps1
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
. "$PSScriptRoot/../end-to-end-tests-prelude.ps1"


$builtinRegistryArgs = $commonArgs + @("--x-builtin-port-versions-dir=$PSScriptRoot/../../e2e_ports/port_versions")
$builtinRegistryArgs = $commonArgs + @("--x-builtin-registry-versions-dir=$PSScriptRoot/../../e2e_ports/versions")

Run-Vcpkg install @builtinRegistryArgs 'vcpkg-internal-e2e-test-port'
Throw-IfNotFailed

# We should not look into the port_versions directory unless we have a baseline,
# We should not look into the versions directory unless we have a baseline,
# even if we pass the registries feature flag
Run-Vcpkg install @builtinRegistryArgs --feature-flags=registries 'vcpkg-internal-e2e-test-port'
Throw-IfNotFailed
Expand All @@ -28,13 +28,13 @@ Copy-Item -Recurse `
-LiteralPath "$PSScriptRoot/../../e2e_ports/vcpkg-internal-e2e-test-port" `
-Destination "$filesystemRegistry"
New-Item `
-Path "$filesystemRegistry/port_versions" `
-Path "$filesystemRegistry/versions" `
-ItemType Directory
Copy-Item `
-LiteralPath "$PSScriptRoot/../../e2e_ports/port_versions/baseline.json" `
-Destination "$filesystemRegistry/port_versions/baseline.json"
-LiteralPath "$PSScriptRoot/../../e2e_ports/versions/baseline.json" `
-Destination "$filesystemRegistry/versions/baseline.json"
New-Item `
-Path "$filesystemRegistry/port_versions/v-" `
-Path "$filesystemRegistry/versions/v-" `
-ItemType Directory

$vcpkgInternalE2eTestPortJson = @{
Expand All @@ -46,7 +46,7 @@ $vcpkgInternalE2eTestPortJson = @{
)
}
New-Item `
-Path "$filesystemRegistry/port_versions/v-/vcpkg-internal-e2e-test-port.json" `
-Path "$filesystemRegistry/versions/v-/vcpkg-internal-e2e-test-port.json" `
-ItemType File `
-Value (ConvertTo-Json -Depth 5 -InputObject $vcpkgInternalE2eTestPortJson)

Expand All @@ -68,7 +68,7 @@ try
$CurrentTest = 'git init .'
git @gitConfigOptions init .
Throw-IfFailed
Copy-Item -Recurse -LiteralPath "$PSScriptRoot/../../e2e_ports/port_versions" -Destination .
Copy-Item -Recurse -LiteralPath "$PSScriptRoot/../../e2e_ports/versions" -Destination .
Copy-Item -Recurse -LiteralPath "$PSScriptRoot/../../e2e_ports/vcpkg-internal-e2e-test-port" -Destination .

$CurrentTest = 'git add -A'
Expand Down
14 changes: 7 additions & 7 deletions scripts/azure-pipelines/end-to-end-tests-dir/versions.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@

# Test verify versions
mkdir $VersionFilesRoot
Copy-Item -Recurse "scripts/testing/version-files/port_versions_incomplete" $VersionFilesRoot
Copy-Item -Recurse "scripts/testing/version-files/versions_incomplete" $VersionFilesRoot
$portsRedirectArgsOK = @(
"--feature-flags=versions",
"--x-builtin-ports-root=scripts/testing/version-files/ports",
"--x-builtin-port-versions-dir=scripts/testing/version-files/port_versions"
"--x-builtin-registry-versions-dir=scripts/testing/version-files/versions"
)
$portsRedirectArgsIncomplete = @(
"--feature-flags=versions",
"--x-builtin-ports-root=scripts/testing/version-files/ports_incomplete",
"--x-builtin-port-versions-dir=$VersionFilesRoot/port_versions_incomplete"
"--x-builtin-registry-versions-dir=$VersionFilesRoot/versions_incomplete"
)
$CurrentTest = "x-verify-ci-versions (All files OK)"
Write-Host $CurrentTest
Expand Down Expand Up @@ -77,22 +77,22 @@ foreach ($opt_registries in @("",",registries"))
./vcpkg $commonArgs "--feature-flags=versions$opt_registries" install `
"--dry-run" `
"--x-manifest-root=scripts/testing/version-files/without-default-baseline-2" `
"--x-builtin-port-versions-dir=scripts/testing/version-files/default-baseline-2/port_versions"
"--x-builtin-registry-versions-dir=scripts/testing/version-files/default-baseline-2/versions"
Throw-IfFailed
Require-FileNotExists $buildtreesRoot/versioning

$CurrentTest = "default baseline 2"
./vcpkg $commonArgs "--feature-flags=versions$opt_registries" install `
"--dry-run" `
"--x-manifest-root=scripts/testing/version-files/default-baseline-2" `
"--x-builtin-port-versions-dir=scripts/testing/version-files/default-baseline-2/port_versions"
"--x-builtin-registry-versions-dir=scripts/testing/version-files/default-baseline-2/versions"
Throw-IfFailed
Require-FileExists $buildtreesRoot/versioning

$CurrentTest = "using version features fails without flag"
./vcpkg $commonArgs "--feature-flags=-versions$opt_registries" install `
"--dry-run" `
"--x-manifest-root=scripts/testing/version-files/default-baseline-2" `
"--x-builtin-port-versions-dir=scripts/testing/version-files/default-baseline-2/port_versions"
"--x-builtin-registry-versions-dir=scripts/testing/version-files/default-baseline-2/versions"
Throw-IfNotFailed
}
}
File renamed without changes.
2 changes: 1 addition & 1 deletion scripts/generateBaseline.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

SCRIPT_DIRECTORY = os.path.dirname(os.path.abspath(__file__))
PORTS_DIRECTORY = os.path.join(SCRIPT_DIRECTORY, '../ports')
VERSIONS_DB_DIRECTORY = os.path.join(SCRIPT_DIRECTORY, '../port_versions')
VERSIONS_DB_DIRECTORY = os.path.join(SCRIPT_DIRECTORY, '../versions')


def get_version_tag(version):
Expand Down
10 changes: 5 additions & 5 deletions scripts/generatePortVersionsDb.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
MAX_PROCESSES = multiprocessing.cpu_count()
SCRIPT_DIRECTORY = os.path.dirname(os.path.abspath(__file__))
PORTS_DIRECTORY = os.path.join(SCRIPT_DIRECTORY, '../ports')
VERSIONS_DB_DIRECTORY = os.path.join(SCRIPT_DIRECTORY, '../port_versions')
VERSIONS_DB_DIRECTORY = os.path.join(SCRIPT_DIRECTORY, '../versions')


def get_current_git_ref():
Expand All @@ -26,7 +26,7 @@ def get_current_git_ref():
return None


def generate_port_versions_file(port_name):
def generate_versions_file(port_name):
containing_dir = os.path.join(VERSIONS_DB_DIRECTORY, f'{port_name[0]}-')
os.makedirs(containing_dir, exist_ok=True)

Expand All @@ -43,7 +43,7 @@ def generate_port_versions_file(port_name):
output.stdout.strip(), file=sys.stderr)


def generate_port_versions_db(revision):
def generate_versions_db(revision):
start_time = time.time()

# Assume each directory in ${VCPKG_ROOT}/ports is a different port
Expand All @@ -54,7 +54,7 @@ def generate_port_versions_db(revision):
concurrency = MAX_PROCESSES / 2
print(f'Running {concurrency:.0f} parallel processes')
process_pool = multiprocessing.Pool(MAX_PROCESSES)
for i, _ in enumerate(process_pool.imap_unordered(generate_port_versions_file, port_names), 1):
for i, _ in enumerate(process_pool.imap_unordered(generate_versions_file, port_names), 1):
sys.stderr.write(
f'\rProcessed: {i}/{total_count} ({(i / total_count):.2%})')
process_pool.close()
Expand All @@ -80,7 +80,7 @@ def main():
print(f'Database files already exist for commit {revision}')
sys.exit(0)

generate_port_versions_db(revision)
generate_versions_db(revision)


if __name__ == "__main__":
Expand Down
1 change: 0 additions & 1 deletion test.cmake

This file was deleted.

4 changes: 2 additions & 2 deletions toolsrc/include/vcpkg/vcpkgcmdarguments.h
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,8 @@ namespace vcpkg
std::unique_ptr<std::string> scripts_root_dir;
constexpr static StringLiteral BUILTIN_PORTS_ROOT_DIR_ARG = "x-builtin-ports-root";
std::unique_ptr<std::string> builtin_ports_root_dir;
constexpr static StringLiteral BUILTIN_PORT_VERSIONS_DIR_ARG = "x-builtin-port-versions-dir";
std::unique_ptr<std::string> builtin_port_versions_dir;
constexpr static StringLiteral BUILTIN_REGISTRY_VERSIONS_DIR_ARG = "x-builtin-registry-versions-dir";
std::unique_ptr<std::string> builtin_registry_versions_dir;

constexpr static StringLiteral DEFAULT_VISUAL_STUDIO_PATH_ENV = "VCPKG_VISUAL_STUDIO_PATH";
std::unique_ptr<std::string> default_visual_studio_path;
Expand Down
2 changes: 1 addition & 1 deletion toolsrc/include/vcpkg/vcpkgpaths.h
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ namespace vcpkg
fs::path scripts;
fs::path prefab;
fs::path builtin_ports;
fs::path builtin_port_versions;
fs::path builtin_registry_versions;

fs::path tools;
fs::path buildsystems;
Expand Down
8 changes: 4 additions & 4 deletions toolsrc/src/vcpkg-test/arguments.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ TEST_CASE ("VcpkgCmdArguments from lowercase argument sequence", "[arguments]")
"C:\\vcpkg",
"--x-scripts-root=C:\\scripts",
"--x-builtin-ports-root=C:\\ports",
"--x-builtin-port-versions-dir=C:\\port_versions",
"--x-builtin-registry-versions-dir=C:\\versions",
"--debug",
"--sendmetrics",
"--printmetrics",
Expand All @@ -28,7 +28,7 @@ TEST_CASE ("VcpkgCmdArguments from lowercase argument sequence", "[arguments]")
REQUIRE(*v.vcpkg_root_dir == "C:\\vcpkg");
REQUIRE(*v.scripts_root_dir == "C:\\scripts");
REQUIRE(*v.builtin_ports_root_dir == "C:\\ports");
REQUIRE(*v.builtin_port_versions_dir == "C:\\port_versions");
REQUIRE(*v.builtin_registry_versions_dir == "C:\\versions");
REQUIRE(v.debug);
REQUIRE(*v.debug.get());
REQUIRE(v.send_metrics);
Expand All @@ -51,7 +51,7 @@ TEST_CASE ("VcpkgCmdArguments from uppercase argument sequence", "[arguments]")
"C:\\vcpkg",
"--X-SCRIPTS-ROOT=C:\\scripts",
"--X-BUILTIN-PORTS-ROOT=C:\\ports",
"--X-BUILTIN-PORT-VERSIONS-DIR=C:\\port_versions",
"--X-BUILTIN-REGISTRY-VERSIONS-DIR=C:\\versions",
"--DEBUG",
"--SENDMETRICS",
"--PRINTMETRICS",
Expand All @@ -64,7 +64,7 @@ TEST_CASE ("VcpkgCmdArguments from uppercase argument sequence", "[arguments]")
REQUIRE(*v.vcpkg_root_dir == "C:\\vcpkg");
REQUIRE(*v.scripts_root_dir == "C:\\scripts");
REQUIRE(*v.builtin_ports_root_dir == "C:\\ports");
REQUIRE(*v.builtin_port_versions_dir == "C:\\port_versions");
REQUIRE(*v.builtin_registry_versions_dir == "C:\\versions");
REQUIRE(v.debug);
REQUIRE(*v.debug.get());
REQUIRE(v.send_metrics);
Expand Down
8 changes: 4 additions & 4 deletions toolsrc/src/vcpkg/commands.add-version.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -334,7 +334,7 @@ namespace vcpkg::Commands::AddVersion
const bool verbose = Util::Sets::contains(parsed_args.switches, OPTION_VERBOSE);

auto& fs = paths.get_filesystem();
auto baseline_path = paths.builtin_port_versions / fs::u8path("baseline.json");
auto baseline_path = paths.builtin_registry_versions / fs::u8path("baseline.json");
if (!fs.exists(VCPKG_LINE_INFO, baseline_path))
{
System::printf(
Expand Down Expand Up @@ -400,8 +400,8 @@ namespace vcpkg::Commands::AddVersion
}
const auto& git_tree = git_tree_it->second;

auto port_versions_path =
paths.builtin_port_versions / Strings::concat(port_name[0], '-') / Strings::concat(port_name, ".json");
auto port_versions_path = paths.builtin_registry_versions / fs::u8path({port_name[0], '-'}) /
fs::u8path(Strings::concat(port_name, ".json"));
update_version_db_file(
paths, port_name, schemed_version, git_tree, port_versions_path, overwrite_version, verbose, add_all);
update_baseline_version(paths, port_name, schemed_version.versiont, baseline_path, verbose);
Expand All @@ -413,4 +413,4 @@ namespace vcpkg::Commands::AddVersion
{
AddVersion::perform_and_exit(args, paths);
}
}
}
6 changes: 3 additions & 3 deletions toolsrc/src/vcpkg/commands.civerifyversions.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -367,8 +367,8 @@ namespace vcpkg::Commands::CIVerifyVersions
continue;
}

auto versions_file_path =
paths.builtin_port_versions / Strings::concat(port_name[0], '-') / Strings::concat(port_name, ".json");
auto versions_file_path = paths.builtin_registry_versions / fs::u8path({port_name[0], '-'}) /
fs::u8path(Strings::concat(port_name, ".json"));
if (!fs.exists(versions_file_path))
{
System::printf(System::Color::error, "FAIL: %s\n", port_name);
Expand Down Expand Up @@ -415,4 +415,4 @@ namespace vcpkg::Commands::CIVerifyVersions
{
CIVerifyVersions::perform_and_exit(args, paths);
}
}
}
Loading

0 comments on commit 68a7495

Please sign in to comment.