Skip to content

tests: Extended Integration Tests - #428

Open
KonradBreitsprecherBkd wants to merge 8 commits into
mainfrom
dev_extend_integrationtests
Open

KonradBreitsprecherBkd wants to merge 8 commits into
mainfrom
dev_extend_integrationtests

Conversation

@KonradBreitsprecherBkd

@KonradBreitsprecherBkd KonradBreitsprecherBkd commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Subject

Tests: extend the integration tests for logging, health monitoring, PCAP tracing, domain sockets, configuration precedence and LIN sleep/wakeup

Description

Adds integration tests for behavior that was so far only checked by hand: end-to-end scenarios with real participants and a registry, which inspect log files, PCAP output and lifecycle states.

New test files:

  • ITest_HealthCheck:
    • A simulation step that exceeds the configured SoftResponseTimeout logs a warning, and the simulation continues.
    • A step that exceeds HardResponseTimeout puts the participant into the Error state.
    • Other participants observe a disconnect.
  • ITest_PcapTracing: Ethernet frames of a controller with a PCAP trace sink end up in a valid PCAP stream, both for a PcapFile and a PcapPipe sink. The stream is parsed without SIL Kit internals, the same way an external tool like Wireshark reads it.
  • ITest_DomainSockets: Participants on the same host connect via local domain sockets by default, and via TCP if domain sockets are disabled.
  • ITest_ConfigPrecedence:
    • The participant name and the registry URI from the participant configuration override the values passed to the API.
    • The API registry URI is used if the configuration has none.
    • The controller network from the configuration overrides the API argument.

Extended:

  • ITest_Lin: new test repeated_sleep_and_wakeup_cycles. LIN master and slave go through several go-to-sleep / wakeup cycles and exchange frames after every wakeup, so a slave that does not return from sleep is detected. The existing sync_lin_simulation only covers a single cycle.
  • ITest_Logging now contains all logging integration tests:
    • Sink levels: each sink only receives messages at or above its level, and Level: Off writes nothing.
    • File sink naming: <LogName>_<ParticipantName>_<Timestamp>.jsonl, with one file per participant.
    • Remote logging, merged from ITest_RemoteLogging, which is removed:
      • the existing two senders / one receiver test
      • LogFromRemotes: false ignores remote messages
      • LogFromRemotes defaults to false
      • the level of the sender's Remote sink filters on the sender side

Test infrastructure:

  • The log file helpers (MakeUniqueLogName, FindLogFiles, ReadTextFile, ScopedLogFiles) are in IntegrationTestUtils.hpp; no new header is added.
  • IntegrationTestUtils.hpp was not included by any test so far. To include it from several tests:
    • its free functions are now inline, which avoids multiple definitions at link time
    • <Windows.h> is included with NOMINMAX
    • platform checks use defined(), because -Wundef failed for __unix__ and WIN32
  • ITest_PcapTracing defines NOMINMAX / WIN32_LEAN_AND_MEAN only if they are not defined yet. MinGW's libstdc++ already defines NOMINMAX, and redefining it broke the MinGW build with -Werror.

Instructions for review / testing

  • Test-only change: the files under SilKit/IntegrationTests/ and their entries in SilKit/IntegrationTests/CMakeLists.txt. No production code changes.
  • Run the new and changed tests:
    SilKitIntegrationTests --gtest_filter=ITest_Logging.*:ITest_HealthCheck.*:ITest_PcapTracing.*:ITest_DomainSockets.*:ITest_ConfigPrecedence*:ITest_LinSleepWakeup.*
    
  • The tests write log and PCAP files with unique names into the working directory and remove them afterwards.
  • The former ITest_RemoteLogging.* tests are now ITest_Logging.*. Update test filters that select them by suite name.
  • Built locally with SILKIT_WARNINGS_AS_ERRORS=ON with MSVC (VS 2026, Debug) and MinGW GCC 13.2 (Ninja). The logging, health check, domain socket and PCAP tests pass with both.

Developer checklist (address before review)

  • Changelog.md updated
  • Prepared update for depending repositories
  • Documentation updated (public API changes only)
  • API docstrings updated (public API changes only)
  • Rebase → commit history clean
  • Squash and merge → proper PR title

@KonradBreitsprecherBkd KonradBreitsprecherBkd changed the title tests: automate manual QA release tests as integration tests tests: Extend integration tests Oct 1, 2026
@KonradBreitsprecherBkd KonradBreitsprecherBkd changed the title tests: Extend integration tests tests: Extended integration tests Oct 1, 2026
@KonradBreitsprecherBkd KonradBreitsprecherBkd changed the title tests: Extended integration tests tests: Extended Integration Tests Oct 1, 2026
added: ITest_HealthCheck for soft/hard response timeouts and disconnects (QA 2.11-2.13)
added: ITest_Logging cases for sink levels and file sink naming (QA 2.6, 2.7)
added: ITest_RemoteLogging cases for LogFromRemotes false/default and remote sink levels (QA 2.8)
added: ITest_PcapTracing for PCAP file and pipe sinks, parsed like an external tool would (QA 2.23.1, 2.23.2)
added: ITest_DomainSockets, same-host connections use local:// unless disabled (QA 2.18, SILKIT-1435)
added: ITest_ConfigPrecedence, configured name, registry URI and network override the API (QA 2.19)
added: ITest_LinSleepWakeup for repeated LIN sleep/wakeup cycles (QA 2.3, SILKIT-1504)
fixed: log file scanning in tests could terminate the test process when parallel tests removed files

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Konrad Breitsprecher <Konrad.Breitsprecher@vector.com>
@KonradBreitsprecherBkd
KonradBreitsprecherBkd force-pushed the dev_extend_integrationtests branch from 88841cb to cb65886 Compare October 2, 2026 07:11
KonradBreitsprecherBkd and others added 4 commits October 2, 2026 09:35
The QA test specification is not part of the public SIL Kit. Describe what the
integration tests check instead of referring to QA test case numbers and
internal issue keys.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Konrad Breitsprecher <Konrad.Breitsprecher@vector.com>
- Move the tests of ITest_RemoteLogging into ITest_Logging, so all logging
  integration tests are in one file and use the ITest_Logging fixture.
- Move the log file helpers of ITestLogFiles.hpp into IntegrationTestUtils.hpp
  instead of adding a new header.
- IntegrationTestUtils.hpp was not included anywhere so far. To include it
  from several tests: make its free functions inline, include <Windows.h> with
  NOMINMAX, and use defined() for platform checks (-Wundef).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Konrad Breitsprecher <Konrad.Breitsprecher@vector.com>
MinGW's libstdc++ already defines NOMINMAX, so defining it again is an error
with -Werror. Define NOMINMAX and WIN32_LEAN_AND_MEAN only if not yet defined.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Konrad Breitsprecher <Konrad.Breitsprecher@vector.com>
Move the repeated sleep/wakeup test into ITest_Lin, next to the single-cycle
sync_lin_simulation test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Konrad Breitsprecher <Konrad.Breitsprecher@vector.com>
@KonradBreitsprecherBkd
KonradBreitsprecherBkd force-pushed the dev_extend_integrationtests branch from 39b89c3 to 1f7ddf2 Compare October 2, 2026 07:35
KonradBreitsprecherBkd and others added 3 commits October 2, 2026 09:43
unlink() is declared in <unistd.h>, which was not included, so the Linux build
failed now that tests include the header. Include the POSIX headers on all
non-Windows platforms, since __unix__ is not defined on macOS.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Konrad Breitsprecher <Konrad.Breitsprecher@vector.com>
On 32-bit builds, std::streamoff returned by tellg() is wider than size_t,
which MSVC reports as C4244 (an error with warnings as errors). Cast it
explicitly.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Konrad Breitsprecher <Konrad.Breitsprecher@vector.com>
Add LogFilePrefix() to IntegrationTestUtils.hpp, so the "<LogName>_<ParticipantName>_" file name scheme is spelled out
in one place instead of appending "_" by hand.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Konrad Breitsprecher <Konrad.Breitsprecher@vector.com>
@KonradBreitsprecherBkd
KonradBreitsprecherBkd force-pushed the dev_extend_integrationtests branch from 6daf1e4 to ac1838c Compare October 2, 2026 12:40
@KonradBreitsprecherBkd
KonradBreitsprecherBkd marked this pull request as ready for review October 2, 2026 13:23
@KonradBreitsprecherBkd KonradBreitsprecherBkd added the needs reviewer This issue is looking for a reviewer. label Oct 2, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs reviewer This issue is looking for a reviewer.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant