Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
changeKind: dependencies
packages:
- "@typespec/http-client-java"
---

Sync Java runtime, test, and Maven plugin dependency versions with Azure SDK for Java.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
changeKind: fix
packages:
- "@typespec/http-client-java"
---

Fix XML pagination with stream-backed responses by caching response bytes before reading items and the next link.
7 changes: 7 additions & 0 deletions .chronus/changes/http-client-java-sync-dependency-versions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
changeKind: internal
packages:
- "@typespec/http-client-java"
---

Add a PowerShell script to sync Java dependency versions with Azure SDK for Java.
7 changes: 4 additions & 3 deletions .github/instructions/http-client-java.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@ Steps:
3. Update package versions in `overrides` (keep the semver ranges) in the other two `package.json` files to match the corresponding package versions in the root `package.json`.
4. Save the files and run `npm install` in the root so that `package-lock.json` is updated.
5. Commit the changes to `package.json` and `package-lock.json`.
6. If there is an update to the `http-specs` or `azure-http-specs` libraries, run `Generate.ps1` in `generator/http-client-generator-test` and commit the generated changes in that folder.
7. If there is an update to the `http-specs` library, run `Generate.ps1` in `generator/http-client-generator-clientcore-test` and commit the generated changes in that folder.
8. Call `pnpm change add @typespec/http-client-java --kind=dependencies --message="<change-summary>"`. Commit the new md file in ".chronus" folder of repository root.
6. Run `pwsh eng\versioning\Sync-Versions.ps1` from the package root to also update Java/JAR dependency versions. Commit the Java and POM changes.
7. If there is an update to the `http-specs` or `azure-http-specs` libraries, run `Generate.ps1` in `generator/http-client-generator-test` and commit the generated changes in that folder.
8. If there is an update to the `http-specs` library, run `Generate.ps1` in `generator/http-client-generator-clientcore-test` and commit the generated changes in that folder.
9. Call `pnpm change add @typespec/http-client-java --kind=dependencies --message="<change-summary>"`. Commit the new md file in ".chronus" folder of repository root.

# Prepare for minor/patch release

Expand Down
12 changes: 12 additions & 0 deletions packages/http-client-java/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,18 @@ When writing / updating Java tests, you can manually
1. Start the test server with `npm run spector-serve` (or `npm run spector-start` to start as a background process, `npm run spector-stop` to stop).
2. Run the Java tests in the module.

## Sync Java Dependency Versions

From this package directory, run `pwsh eng\versioning\Sync-Versions.ps1` to sync
`Project.Dependency`, generator/module POMs, the customization POM, and the generated-POM
template with Azure SDK for Java's version files. Use `-WhatIf` to preview changes.

`eng\versioning\external_dependencies.txt` lists the packages to sync; append `;version`
to pin a package. The script prefers the first (released) version in `version_client.txt`,
then falls back to `external_dependencies.txt`. Unlisted dependencies are unchanged.

Run the script's offline tests with `pwsh eng\versioning\Sync-Versions.Tests.ps1`.

## Format Code

Before creating pull request, the code is required to be well-formatted.
Expand Down
135 changes: 135 additions & 0 deletions packages/http-client-java/eng/versioning/Sync-Versions.Tests.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.

$ErrorActionPreference = 'Stop'
. (Join-Path $PSScriptRoot 'Sync-Versions.ps1')

function Assert-Equal($Expected, $Actual)
{
if ($Expected -cne $Actual)
{
throw "Expected '$Expected', got '$Actual'."
}
}

$clientVersions = Read-Versions @'
# Released version precedes the development version.
com.azure:azure-core;1.2.3;1.3.0-beta.1
com.azure.v2:azure-client-sdk-parent;2.0.0-beta.2;2.0.0-beta.3
io.clientcore:core;1.0.0-beta.11;1.0.0-beta.12
'@
$externalVersions = Read-Versions @'
com.azure:azure-core;0.9.0
testdep_net.bytebuddy:byte-buddy;1.17.7
org.mockito:mockito-core;5.0.0
'@
$packages = Read-Versions @'
com.azure:azure-core
com.azure.v2:azure-client-sdk-parent
io.clientcore:core
net.bytebuddy:byte-buddy
org.mockito:mockito-core;4.11.0
'@
$versions = Resolve-Versions $packages @($clientVersions, $externalVersions)
Assert-Equal '1.2.3' $versions['com.azure:azure-core']
Assert-Equal '1.0.0-beta.11' $versions['io.clientcore:core']
Assert-Equal '1.17.7' $versions['net.bytebuddy:byte-buddy']
Assert-Equal '4.11.0' $versions['org.mockito:mockito-core']

$failed = $false
try
{
Resolve-Versions (Read-Versions 'missing:dependency') @($clientVersions, $externalVersions)
}
catch
{
if ($_.Exception.Message -notlike '*missing:dependency*') { throw }
$failed = $true
}
Assert-Equal $true $failed

$root = Join-Path ([System.IO.Path]::GetTempPath()) "java-version-sync-$([guid]::NewGuid())"
try
{
$generator = Join-Path $root 'generator'
$module = Join-Path $generator 'http-client-generator-core'
$resources = Join-Path $module 'src\main\resources'
$javaDirectory = Join-Path $module 'src\main\java\com\microsoft\typespec\http\client\generator\core\model\projectmodel'
$target = Join-Path $module 'target'
$customization = Join-Path $generator 'http-client-generator-test\customization'
foreach ($directory in @($resources, $javaDirectory, $target, $customization))
{
$null = New-Item -ItemType Directory -Path $directory -Force
}
$pom = @'
<project>
<!-- <groupId>com.azure</groupId><artifactId>azure-core</artifactId><version>comment</version> -->
<parent>
<groupId>com.microsoft.typespec</groupId>
<artifactId>typespec-java-generator</artifactId>
<version>1.0.0-beta.1</version>
</parent>
<dependencies>
<dependency>
<groupId>com.azure</groupId>
<!-- Keep this comment and the original formatting. -->
<artifactId>azure-core</artifactId>
<version>old</version>
<exclusions>
<exclusion><groupId>io.clientcore</groupId><artifactId>core</artifactId></exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>unlisted</groupId><artifactId>library</artifactId><version>old</version>
</dependency>
<dependency>
<groupId>org.mockito</groupId><artifactId>mockito-core</artifactId><version>old</version>
</dependency>
</dependencies>
</project>
'@
$pom = $pom.Replace("`r`n", "`n").Replace("`n", "`r`n")
$pomFiles = @(
(Join-Path $generator 'pom.xml'),
(Join-Path $module 'pom.xml'),
(Join-Path $resources 'pom.xml'),
(Join-Path $customization 'pom.xml')
)
foreach ($file in $pomFiles + (Join-Path $target 'pom.xml'))
{
[System.IO.File]::WriteAllText($file, $pom)
}
$javaFile = Join-Path $javaDirectory 'Project.java'
$java = " AZURE_CORE(`"com.azure`", `"azure-core`", `"old`"),`n" +
" AZURE_CLIENT_SDK_PARENT_V2(`"com.azure.v2`", `"azure-client-sdk-parent`", `"old-v2`"),`n" +
" CLIENTCORE(`"io.clientcore`", `"core`", `"old`");`n"
[System.IO.File]::WriteAllText($javaFile, $java)

Update-VersionFiles -Root $root -Versions $versions -WhatIf
Assert-Equal $pom ([System.IO.File]::ReadAllText($pomFiles[0]))
Assert-Equal $java ([System.IO.File]::ReadAllText($javaFile))

Update-VersionFiles -Root $root -Versions $versions
$expected = $pom.Replace(" <version>old</version>", " <version>1.2.3</version>")
$expected = $expected.Replace('<artifactId>mockito-core</artifactId><version>old</version>',
'<artifactId>mockito-core</artifactId><version>4.11.0</version>')
foreach ($file in $pomFiles)
{
Assert-Equal $expected ([System.IO.File]::ReadAllText($file))
}
Assert-Equal $pom ([System.IO.File]::ReadAllText((Join-Path $target 'pom.xml')))
Assert-Equal ($java.Replace('"old"),', '"1.2.3"),').Replace('"old");', '"1.0.0-beta.11");').Replace('"old-v2"', '"2.0.0-beta.2"')) `
([System.IO.File]::ReadAllText($javaFile))

$modified = (Get-Item $javaFile).LastWriteTimeUtc
Update-VersionFiles -Root $root -Versions $versions
Assert-Equal $modified (Get-Item $javaFile).LastWriteTimeUtc
Write-Host 'Version sync tests passed.'
}
finally
{
if (Test-Path -LiteralPath $root)
{
Remove-Item -LiteralPath $root -Recurse -Force
}
}
110 changes: 110 additions & 0 deletions packages/http-client-java/eng/versioning/Sync-Versions.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.

[CmdletBinding(SupportsShouldProcess)]
param()

$ErrorActionPreference = 'Stop'

function Read-Versions([string] $Content)
{
$versions = @{}
foreach ($line in $Content -split '\r?\n')
{
$line = $line.Trim()
if (!$line -or $line.StartsWith('#')) { continue }
$fields = $line -split ';'
$package = $fields[0] -replace '^testdep_', ''
if ($package -notmatch '^[\w.-]+:[\w.-]+$')
{
throw "Invalid package coordinate: $line"
}
$versions[$package] = if ($fields.Length -gt 1) { $fields[1].Trim() } else { '' }
}
return $versions
}

function Resolve-Versions([hashtable] $Packages, [hashtable[]] $Sources)
{
$versions = @{}
foreach ($package in $Packages.Keys)
{
$version = $Packages[$package]
if (!$version)
{
foreach ($source in $Sources)
{
if ($source[$package])
{
$version = $source[$package]
break
}
}
}
if (!$version) { throw "Version not found for package $package" }
$versions[$package] = $version
}
return $versions
}

function Update-VersionFiles
{
[CmdletBinding(SupportsShouldProcess)]
param([string] $Root, [hashtable] $Versions)

$generator = Join-Path $Root 'generator'
$core = Join-Path $generator 'http-client-generator-core'
$pomFiles = @((Get-Item (Join-Path $generator 'pom.xml')).FullName)
$pomFiles += Get-ChildItem -Path $generator -Directory | ForEach-Object {
$pom = Join-Path $_.FullName 'pom.xml'
if (Test-Path -LiteralPath $pom) { $pom }
}
$pomFiles += Join-Path $core 'src\main\resources\pom.xml'
$pomFiles += Join-Path $generator 'http-client-generator-test\customization\pom.xml'
$javaFile = Join-Path $core 'src\main\java\com\microsoft\typespec\http\client\generator\core\model\projectmodel\Project.java'
$updates = @{}
foreach ($file in $pomFiles + $javaFile)
{
$content = [System.IO.File]::ReadAllText($file)
if ($file -eq $javaFile)
{
$pattern = '(?m)(?<prefix>^\s*[A-Z][A-Z_0-9]*\("(?<group>[^"]+)",\s*"(?<artifact>[^"]+)",\s*")(?<version>[^"]+)(?<suffix>"\)[,;])'
}
else
{
# Match comments separately so commented-out dependencies are never changed.
$gap = '(?:\s|<!--.*?-->)*'
$pattern = "<!--.*?-->|(?<prefix><groupId>(?<group>[^<]+)</groupId>$gap<artifactId>(?<artifact>[^<]+)</artifactId>$gap<version>)(?<version>[^<]+)(?<suffix></version>)"
$null = [xml] $content
}
$updated = [regex]::Replace($content, $pattern, [System.Text.RegularExpressions.MatchEvaluator] {
param($match)
if (!$match.Groups['prefix'].Success) { return $match.Value }
$package = "$($match.Groups['group'].Value.Trim()):$($match.Groups['artifact'].Value.Trim())"
if (!$Versions.ContainsKey($package)) { return $match.Value }
return $match.Groups['prefix'].Value + $Versions[$package] + $match.Groups['suffix'].Value
}, [System.Text.RegularExpressions.RegexOptions]::Singleline)
if ($updated -cne $content) { $updates[$file] = $updated }
}

# Read and validate all inputs before writing any changes.
foreach ($file in $updates.Keys | Sort-Object)
{
if ($PSCmdlet.ShouldProcess($file, 'Sync dependency versions'))
{
[System.IO.File]::WriteAllText($file, $updates[$file])
Write-Host "Updated $file"
}
}
}

if ($MyInvocation.InvocationName -ne '.')
{
$root = [System.IO.Path]::GetFullPath((Join-Path $PSScriptRoot '..\..'))
$baseUri = 'https://raw.githubusercontent.com/Azure/azure-sdk-for-java/main/eng/versioning'
$clientVersions = Read-Versions (Invoke-WebRequest "$baseUri/version_client.txt").Content
$externalVersions = Read-Versions (Invoke-WebRequest "$baseUri/external_dependencies.txt").Content
$packages = Read-Versions ([System.IO.File]::ReadAllText((Join-Path $PSScriptRoot 'external_dependencies.txt')))
$versions = Resolve-Versions $packages @($clientVersions, $externalVersions)
Update-VersionFiles -Root $root -Versions $versions -WhatIf:$WhatIfPreference
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Packages to sync from azure-sdk-for-java. Add ;version to pin a package.
com.azure:azure-client-sdk-parent
com.azure.v2:azure-client-sdk-parent
com.azure.v2:azure-core
com.azure:azure-core
com.azure:azure-core-experimental
com.azure:azure-core-http-netty
com.azure:azure-core-management
com.azure:azure-core-test
com.azure:azure-identity
com.azure:azure-json
com.azure:azure-xml

io.clientcore:core
io.clientcore:annotation-processor

com.fasterxml.jackson.core:jackson-databind
com.fasterxml.jackson.dataformat:jackson-dataformat-xml
com.fasterxml.jackson.datatype:jackson-datatype-jsr310
org.slf4j:slf4j-api
org.slf4j:slf4j-simple
org.junit.jupiter:junit-jupiter-api
org.junit.jupiter:junit-jupiter-engine
org.junit.jupiter:junit-jupiter-params
io.projectreactor:reactor-test

# Mockito 4.11.0 is the last version supporting Java 8.
org.mockito:mockito-core;4.11.0
net.bytebuddy:byte-buddy
net.bytebuddy:byte-buddy-agent

org.apache.maven.plugins:maven-clean-plugin
org.apache.maven.plugins:maven-compiler-plugin
org.apache.maven.plugins:maven-install-plugin
org.apache.maven.plugins:maven-jar-plugin
org.apache.maven.plugins:maven-javadoc-plugin
org.apache.maven.plugins:maven-resources-plugin
org.apache.maven.plugins:maven-shade-plugin
org.apache.maven.plugins:maven-surefire-plugin
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-test</artifactId>
<version>3.7.18</version>
<version>3.8.7</version>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
Expand All @@ -47,13 +47,13 @@
<dependency>
<groupId>net.bytebuddy</groupId>
<artifactId>byte-buddy</artifactId>
<version>1.17.7</version>
<version>1.18.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>net.bytebuddy</groupId>
<artifactId>byte-buddy-agent</artifactId>
<version>1.17.7</version>
<version>1.18.11</version>
<scope>test</scope>
</dependency>
<dependency>
Expand All @@ -80,7 +80,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.14.0</version>
<version>3.15.0</version>
<executions>
<execution>
<id>run-annotation-processing</id>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-core</artifactId>
<version>1.58.1</version>
<version>1.60.0</version>
<scope>test</scope>
</dependency>
<dependency>
Expand Down
Loading
Loading