From 65782463ea833bf4a3615d865d73f3a0b8916594 Mon Sep 17 00:00:00 2001 From: Sufiyan Date: Tue, 25 Nov 2025 23:34:53 +0000 Subject: [PATCH 1/6] Add matchers documentation to service_virtualization --- .../service_virtualization.md | 624 ++++++++++++++++-- 1 file changed, 559 insertions(+), 65 deletions(-) diff --git a/contract_driven_development/service_virtualization.md b/contract_driven_development/service_virtualization.md index 16f4680e5..55723c345 100644 --- a/contract_driven_development/service_virtualization.md +++ b/contract_driven_development/service_virtualization.md @@ -13,71 +13,103 @@ Service Virtualization Mocking -* [Service Virtualization](#service-virtualization) - * [Pre-requisites](#pre-requisites) - * [Inline Examples](#inline-examples) - * [Inline Examples for Responses with No Body](#inline-examples-for-responses-with-no-body) - * [Example Usage](#example-usage) - * [Key Points:](#key-points) - * [Externalizing Example Data](#externalizing-example-data) - * [Handling No Response Body APIs](#handling-no-response-body-apis) - * [Intelligent Service Virtualization - Example cannot go out of sync](#intelligent-service-virtualization---example-cannot-go-out-of-sync) - * [Strict Mode](#strict-mode) - * [Data Type-Based Examples](#data-type-based-examples) - * [Plain Text Request Bodies - Examples With Regular Expressions](#plain-text-request-bodies---examples-with-regular-expressions) - * [Correlated Request And Response Values](#correlated-request-and-response-values) - * [Direct Substitution - Copying Values From Request To Response](#direct-substitution---copying-values-from-request-to-response) - * [Data Lookup](#data-lookup) - * [Partial Examples](#partial-examples) - * [Partial Request Examples](#partial-request-examples) - * [Partial Response Examples](#partial-response-examples) - * [Use Meaningful Response Values From An External Dictionary](#use-meaningful-response-values-from-an-external-dictionary) - * [Delay Simulation](#delay-simulation) - * [Example Specific Delay](#example-specific-delay) - * [Global Delay](#global-delay) - * [SSL / HTTPS Stubbing](#ssl--https--stubbing) - * [Auto-Generated Cert Store](#auto-generated-cert-store) - * [Bring Your Own Key Store](#bring-your-own-key-store) - * [Dynamic Expectations (a.k.a. Dynamic Stubbing Or Mocking) - Setting Expectations Over Specmatic Http Api](#dynamic-expectations-aka-dynamic-stubbing-or-mocking---setting-expectations-over-specmatic-http-api) - * [Context](#context) - * [Expectations Http Endpoint](#expectations-http-endpoint) - * [Anatomy of a Component / API Test](#anatomy-of-a-component--api-test) - * [Programmatically Starting Stub Server Within Tests](#programmatically-starting-stub-server-within-tests) - * [Transient Expectations (a.k.a. Transient Stubs)](#transient-expectations-aka-transient-stubs) - * [Setting transient expectations](#setting-transient-expectations) - * [Clearing Transient Expectations](#clearing-transient-expectations) - * [Externalised Response Generation](#externalised-response-generation) - * [Hooks](#hooks) - * [Overview](#overview) - * [Use Case: API Gateway Header Transformation](#use-case-api-gateway-header-transformation) - * [Implementation Steps](#implementation-steps) - * [How It Works](#how-it-works) - * [Precedence Across Types Of Examples](#precedence-across-types-of-examples) - * [Checking Health Status Of Stub Server](#checking-health-status-of-stub-server) - * [Example `curl` Request:](#example-curl-request) - * [Specmatic Configuration with Base URL, Host, Port, and Path](#specmatic-configuration-with-base-url-host-port-and-path) - * [Specifying Host](#specifying-host) - * [Specifying Port](#specifying-port) - * [Specifying Base Path](#specifying-base-path) - * [Specifying Combination of Host, Port, and Base Path](#specifying-combination-of-host-port-and-base-path) - * [Specifying Base URL](#specifying-base-url) - * [Customizing the Default Base URL](#customizing-the-default-base-url) - * [Running Specmatic Stubs on Multiple BaseURLs](#running-specmatic-stubs-on-multiple-baseurls) - * [Overview](#overview-1) - * [Directory Structure](#directory-structure) - * [Specmatic Configuration](#specmatic-configuration) - * [API Specifications](#api-specifications) - * [imported_product.yaml](#imported_productyaml) - * [exported_product.yaml](#exported_productyaml) - * [Examples](#examples) - * [post_imported_product.json](#post_imported_productjson) - * [post_exported_product.json](#post_exported_productjson) - * [Run the stub server](#run-the-stub-server) - * [Example Requests](#example-requests) - * [Benefits](#benefits) - * [Disable hot-reload](#disable-hot-reload) - * [Using matching branches in the central contract repo](#using-matching-branches-in-the-central-contract-repo) - * [Sample Java Project](#sample-java-project) +- [Service Virtualization](#service-virtualization) + - [Pre-requisites](#pre-requisites) + - [Inline Examples](#inline-examples) + - [Inline Examples for Responses with No Body](#inline-examples-for-responses-with-no-body) + - [Example Usage](#example-usage) + - [Key Points:](#key-points) + - [Externalizing Example Data](#externalizing-example-data) + - [Handling No Response Body APIs](#handling-no-response-body-apis) + - [Intelligent Service Virtualization - Example cannot go out of sync](#intelligent-service-virtualization---example-cannot-go-out-of-sync) + - [Strict Mode](#strict-mode) + - [Data Type-Based Examples](#data-type-based-examples) + - [Plain Text Request Bodies - Examples With Regular Expressions](#plain-text-request-bodies---examples-with-regular-expressions) + - [Correlated Request And Response Values](#correlated-request-and-response-values) + - [Direct Substitution - Copying Values From Request To Response](#direct-substitution---copying-values-from-request-to-response) + - [Data Lookup](#data-lookup) + - [Partial Examples](#partial-examples) + - [Partial Request Examples](#partial-request-examples) + - [Partial Response Examples](#partial-response-examples) + - [Use Meaningful Response Values From An External Dictionary](#use-meaningful-response-values-from-an-external-dictionary) + - [Delay Simulation](#delay-simulation) + - [Example Specific Delay](#example-specific-delay) + - [Global Delay](#global-delay) + - [Request Matchers](#request-matchers) + - [Why Use Matchers?](#why-use-matchers) + - [Key Benefits](#key-benefits) + - [Equality Matcher: `$eq`](#equality-matcher-eq) + - [Matcher Reference](#matcher-reference) + - [Configuration Keys](#configuration-keys) + - [Example](#example) + - [Inequality Matcher: `$neq`](#inequality-matcher-neq) + - [Matcher Reference](#matcher-reference-1) + - [Configuration Keys](#configuration-keys-1) + - [Example](#example-1) + - [Pattern Matcher: `$pattern`](#pattern-matcher-pattern) + - [Matcher Reference](#matcher-reference-2) + - [Configuration Keys](#configuration-keys-2) + - [Matching Modes](#matching-modes) + - [Example 1: Built-in Type Validation](#example-1-built-in-type-validation) + - [Example 2: Custom Schema with Partial Matching](#example-2-custom-schema-with-partial-matching) + - [Repetition Matcher: `$repeat`](#repetition-matcher-repeat) + - [Matcher Reference](#matcher-reference-3) + - [Configuration Keys](#configuration-keys-3) + - [Counting Strategies](#counting-strategies) + - [Stub Exhaustion Rules](#stub-exhaustion-rules) + - [Example 1: Total Volume Repetition](#example-1-total-volume-repetition) + - [Example 2: Unique Value Repetition](#example-2-unique-value-repetition) + - [Example 3: Combined Repetition Strategies](#example-3-combined-repetition-strategies) + - [Composite Matcher: `$match`](#composite-matcher-match) + - [Matcher Reference](#matcher-reference-4) + - [Using Multiple Matchers Together](#using-multiple-matchers-together) + - [Example](#example-2) + - [Simulating Downstream Dependency Failures using matchers](#simulating-downstream-dependency-failures-using-matchers) + - [First 2 Requests - Simulated Latency](#first-2-requests---simulated-latency) + - [Requests After Exhaustion - Recovery:](#requests-after-exhaustion---recovery) + - [SSL / HTTPS Stubbing](#ssl--https--stubbing) + - [Auto-Generated Cert Store](#auto-generated-cert-store) + - [Bring Your Own Key Store](#bring-your-own-key-store) + - [Dynamic Expectations (a.k.a. Dynamic Stubbing Or Mocking) - Setting Expectations Over Specmatic Http Api](#dynamic-expectations-aka-dynamic-stubbing-or-mocking---setting-expectations-over-specmatic-http-api) + - [Context](#context) + - [Expectations Http Endpoint](#expectations-http-endpoint) + - [Anatomy of a Component / API Test](#anatomy-of-a-component--api-test) + - [Programmatically Starting Stub Server Within Tests](#programmatically-starting-stub-server-within-tests) + - [Transient Expectations (a.k.a. Transient Stubs)](#transient-expectations-aka-transient-stubs) + - [Setting transient expectations](#setting-transient-expectations) + - [Clearing Transient Expectations](#clearing-transient-expectations) + - [Externalised Response Generation](#externalised-response-generation) + - [Hooks](#hooks) + - [Overview](#overview) + - [Use Case: API Gateway Header Transformation](#use-case-api-gateway-header-transformation) + - [Implementation Steps](#implementation-steps) + - [How It Works](#how-it-works) + - [Precedence Across Types Of Examples](#precedence-across-types-of-examples) + - [Checking Health Status Of Stub Server](#checking-health-status-of-stub-server) + - [Example `curl` Request:](#example-curl-request) + - [Specmatic Configuration with Base URL, Host, Port, and Path](#specmatic-configuration-with-base-url-host-port-and-path) + - [Specifying Host](#specifying-host) + - [Specifying Port](#specifying-port) + - [Specifying Base Path](#specifying-base-path) + - [Specifying Combination of Host, Port, and Base Path](#specifying-combination-of-host-port-and-base-path) + - [Specifying Base URL](#specifying-base-url) + - [Customizing the Default Base URL](#customizing-the-default-base-url) + - [Running Specmatic Stubs on Multiple BaseURLs](#running-specmatic-stubs-on-multiple-baseurls) + - [Overview](#overview-1) + - [Directory Structure](#directory-structure) + - [Specmatic Configuration](#specmatic-configuration) + - [API Specifications](#api-specifications) + - [imported\_product.yaml](#imported_productyaml) + - [exported\_product.yaml](#exported_productyaml) + - [Examples](#examples) + - [post\_imported\_product.json](#post_imported_productjson) + - [post\_exported\_product.json](#post_exported_productjson) + - [Run the stub server](#run-the-stub-server) + - [Example Requests](#example-requests) + - [Benefits](#benefits) + - [Disable hot-reload](#disable-hot-reload) + - [Using matching branches in the central contract repo](#using-matching-branches-in-the-central-contract-repo) + - [Sample Java Project](#sample-java-project) ## Pre-requisites @@ -1056,6 +1088,468 @@ stub: **Note:** If the delay is specified in the example file, it will be used to simulate response times for that specific example. Otherwise, the global delay will be applied. +## Request Matchers + +Specmatic provides a range of matchers that can be used to compare incoming requests with predefined examples and deliver the appropriate responses + +These matchers can match `data types`, `exact values`, and `repetitions`, etc, and Specmatic will only respond with the corresponding response if all the matchers are satisfied + +{:.note} +> It is critical to understand that Specmatic enforces validations in a specific order: +> - **OpenAPI Spec Validation**: First, the request is validated against your OpenAPI specification. If a field is marked as required in the spec, it must be present and valid +> - **Matcher Validation**: The custom matcher (like `$pattern` or `$match`) only execute after the request has satisfied the base contract + +### Why Use Matchers? + +Matchers provide fine-grained control over how your stub server responds to incoming requests. Instead of relying solely on static examples, matchers enable dynamic, stateful, conditional response selection based on request content. + +#### Key Benefits + +- **Stateful Behavior**: Track request counts and simulate exhaustion for advanced scenarios +- **Resilience**: Simulate downstream service behaviors including failures, delays, and edge cases +- **Flexible Request Matching**: Respond differently based on field values, types, or patterns, etc. +- **Schema Validation**: Enforce incoming requests to conform to sub-schemas before returning responses + +### Equality Matcher: `$eq` + +**Purpose**: Validate exact equality of a field value in incoming requests + +#### Matcher Reference + +| Attribute | Value | +| ---------------- | ------------------------------------------------------------------------------- | +| Syntax | `$eq(Value)` | +| Description | Succeeds only if the incoming request value is identical to the specified value | +| Supported Types | Strings, numbers, and other data types | +| Case Sensitivity | Yes | + + +#### Configuration Keys + +| Key | Description | +| ----------- | --------------------------------- | +| `exact` | The exact value to match | +| `matchType` | MatchType defaults to eq (eqauls) | + + +#### Example + +**Requirement**: Confirm that the incoming user's role is strictly `Admin`, +The stub responds with HTTP 200 OK only if the matcher is satisfied. + +```json +{ + "http-request": { + "method": "POST", + "path": "/verifyUser", + "body": { + "role": "$eq(Admin)" + } + }, + "http-response": { + "status": 200, + "status-text": "OK" + } +} +``` + +### Inequality Matcher: `$neq` + +**Purpose**: Validate that a field value in incoming requests differs from a specified value + +#### Matcher Reference + +| Attribute | Value | +| ---------------- | ----------------------------------------------------------------------- | +| Syntax | `$neq(Value)` | +| Description | Succeeds if the incoming request value differs from the specified value | +| Supported Types | Strings, numbers, and other data types | +| Case Sensitivity | Yes | + +#### Configuration Keys + +| Key | Description | +| ----------- | --------------------------------------- | +| `exact` | The value to not match | +| `matchType` | Set to `neq` (not equal) for inequality | + +#### Example + +**Requirement**: Confirm that the incoming request status is NOT `Pending`, The stub responds with HTTP 200 OK only if the matcher is satisfied. + +```json +{ + "http-request": { + "method": "POST", + "path": "/verifyUser", + "body": { + "status": "$neq(Pending)" + } + }, + "http-response": { + "status": 200, + "status-text": "OK" + } +} +``` + +### Pattern Matcher: `$pattern` + +**Purpose**: Validate that a field's structure in incoming requests matches a specific schema (built-in types or custom schemas). + +#### Matcher Reference + +| Attribute | Value | +| ------------------ | ------------------------------------------------------------------------------------------------------ | +| Syntax | `$pattern(Schema)` or `$pattern(dataType: Schema, partial: boolean)` | +| Description | Validates incoming request data against a schema definition with configurable partial matching support | +| Supported Patterns | Built-in types (`uuid`, `email`, `date`, etc.) and custom schemas from your specification | +| Matching Modes | Strict or Partial | + +#### Configuration Keys + +| Key | Description | Values | +| ---------- | -------------------------------------- | --------------------------------------------- | +| `dataType` | The schema name to validate against | Built-in type or custom schema name | +| `partial` | Allow missing fields during validation | `true` (partial) or `false` (strict, default) | + +#### Matching Modes + +| Mode | Behavior | When to Use | +| ----------- | ---------------------------------------------------------------------------------------- | ----------------------------------------------------- | +| **Strict** | Incoming data must contain all required fields. Missing fields cause validation failure. | When exact schema compliance is required | +| **Partial** | Keys present must match the schema, but missing keys are allowed. | When validating subset of data (e.g., PATCH requests) | + +#### Example 1: Built-in Type Validation + +**Requirement**: Ensure that the incoming payment request contains a valid UUID for the `transactionId`, The stub only responds with `processing` if the matcher is satisfied. + +```json +{ + "http-request": { + "method": "POST", + "path": "/processPayment", + "body": { + "amount": 100, + "transactionId": "$pattern(uuid)" + } + }, + "http-response": { + "status": 200, + "body": { + "status": "processing" + } + } +} +``` + +#### Example 2: Custom Schema with Partial Matching + +**Requirement**: Validate an incoming `PATCH` request where only some contact fields may be updated. +The stub should accept partial updates that match the `ContactInfo` schema structure. + +**API Specification**: + +```yaml +paths: + '/users/{id}': + patch: + summary: Update user details + requestBody: + content: + application/json: + schema: + type: object + properties: + contact: + type: object +components: + schemas: + ContactInfo: + type: object + required: + - email + - phone + - city + properties: + email: + type: string + format: email + phone: + type: string + city: + type: string +``` + +**Example File**: + +```json +{ + "http-request": { + "method": "PATCH", + "path": "/users/123", + "body": { + "contact": "$pattern(dataType: ContactInfo, partial: true)" + } + }, + "http-response": { + "status": 200, + "status-text": "Updated" + } +} +``` + +**How this works**: +1. **Spec Validation**: The PATCH endpoint allows `contact` as a flexible object, so this passes. +2. **Matcher Validation**: The matcher validates that fields in the incoming `contact` object conform to `ContactInfo`, Missing fields are **not** treated as failures because `partial: true` is specified. + +### Repetition Matcher: `$repeat` + +**Purpose**: Track and limit how many times a specific example response can be returned across multiple incoming requests (stateful matcher), For this to work the example must be marked as `transient` + +#### Matcher Reference + +| Attribute | Value | +| ----------- | ------------------------------------------------------------------------------------- | +| Syntax | `$repeat(times: N, value: Strategy)` | +| Description | Limits the reusability of an example based on incoming request count or unique values | +| Stateful | Yes (tracks across multiple requests) | +| Exhaustion | Example becomes unavailable after matcher conditions are met | + +#### Configuration Keys + +| Key | Description | Values | +| ------- | ---------------------------------------------------------- | --------------------------------------------- | +| `value` | Counting strategy for determining usage | `any` (total count) or `each` (unique values) | +| `times` | Number of times the matcher allows reuse before exhaustion | Integer (e.g., `3`, `5`, `10`) | + +#### Counting Strategies + +| Strategy | Logic | Example | +| -------- | ------------------------------------------------------------------------------------------ | ------------------------------------------ | +| `any` | Every incoming matching request consumes 1 count, irrespective of data content | Three requests → count reaches 3 | +| `each` | Only unique values in incoming requests consume a count; identical values do not increment | Three identical requests → count reaches 1 | + +#### Stub Exhaustion Rules + +An example is considered **exhausted** when: +1. All repetition matchers within the example have met their `times` limit in accordance with their `value` strategy +2. Combination of the repetition matchers has also been exhausted `N` times, where `N` is the maximum of all repetition matchers within the example + +Once exhausted, Specmatic stops returning this example and moves to the next matching example. + +#### Example 1: Total Volume Repetition + +**Requirement**: The `/status` endpoint should return the same response for the first 3 incoming requests (total volume count), +After 3 requests, the example is exhausted and the next matching example is used. + +```json +{ + "transient": true, + "http-request": { + "method": "GET", + "path": "/status", + "body": { + "requestId": "$repeat(times: 3, value: any)" + } + }, + "http-response": { + "status": 200, + "body": { + "message": "Service operational" + } + } +} +``` + +**Request Sequence**: +- Request 1 → Returns response (count: 1) +- Request 2 → Returns response (count: 2) +- Request 3 → Returns response (count: 3, **exhausted**) +- Request 4 → Uses next matching example + +#### Example 2: Unique Value Repetition + +**Requirement**: The `/user/{userId}` endpoint should return the same response once per unique `userId` in incoming requests, +After 5 unique userIds are seen, the example is exhausted. + +```json +{ + "transient": true, + "http-request": { + "method": "GET", + "path": "/user/(number)", + "body": { + "userId": "$repeat(times: 5, value: each)" + } + }, + "http-response": { + "status": 200, + "body": { + "name": "John Doe" + } + } +} +``` + +**Request Sequence**: +- Request 1 (userId: 1) → Returns response (unique count: 1) +- Request 2 (userId: 1) → Returns response (unique count: 1, no increment) +- Request 3 (userId: 2) → Returns response (unique count: 2) +- Request 4 (userId: 2) → Returns response (unique count: 2, no increment) +- Request 5 (userId: 3) → Returns response (unique count: 3) +- Request 6 (userId: 4) → Returns response (unique count: 4) +- Request 7 (userId: 5) → Returns response (unique count: 5, **exhausted**) +- Request 8 (userId: 6) → Uses next matching example + +#### Example 3: Combined Repetition Strategies + +**Requirement**: The `/validate` endpoint contains two repeating fields with different constraints: +1. `category`: Should match up to 2 unique values (value: each) +2. `requestId`: Should match up to 4 total requests (value: any) + +**Exhaustion Rule**: The example is considered exhausted when the request count reaches 4 (the maximum of 2 and 4). Until that limit is reached, the stub remains active provided incoming requests satisfy both local matchers and the overall exhaustion rule. Note that individual matchers may reach their limits earlier than the overall exhaustion point. + +```json +{ + "transient": true, + "http-request": { + "method": "POST", + "path": "/validate", + "body": { + "category": "$repeat(times: 2, value: each)", + "requestId": "$repeat(times: 4, value: any)" + } + }, + "http-response": { + "status": 202, + "body": { + "status": "Accepted" + } + } +} +``` + +**Request Sequence**: +- Request 1 (`category`: "A", `requestId`: "101") +- → Returns 202 (category count: 1/2, requestId count: 1/4, **Active**) +- Request 2 (`category`: "B", `requestId`: "102") +- → Returns 202 (category count: 2/2, requestId count: 2/4, **Active**) +- Request 3 (`category`: "C", `requestId`: "103") +- → Returns 202 (category count: 2/2 - *Local Limit Met*, requestId count: 3/4, **Active**) +- Request 4 (`category`: "D", `requestId`: "104") +- → Returns 202 (category count: 2/2 - *Local Limit Met*, requestId count: 4/4, **Exhausted**) +- Request 5 (`category`: "E", `requestId`: "105") → Uses next matching example (first example is fully exhausted) + +### Composite Matcher: `$match` + +**Purpose**: Combine multiple validation rules into a single flexible matcher configuration for incoming requests. + +#### Matcher Reference + +| Attribute | Value | +| ---------------------- | --------------------------------------------------------------------------------------------------------- | +| Syntax | `$match(key1: value1, key2: value2, ...)` | +| Description | Combines equality, type validation, partial matching, and/or repetition constraints for incoming requests | +| Supported Combinations | Any compatible combination of configuration keys | +| Flexibility | Highly flexible for complex scenarios | + +#### Using Multiple Matchers Together + +The `$match` composite matcher allows you to combine validation capabilities from any of the individual matchers (`$eq`, `$neq`, `$pattern`, `$repeat`) within a single field. Simply include the configuration keys you need from the matchers you want to use. + +For example: +- To combine `$pattern` and `$repeat`: use `dataType` and `times` / `value` keys +- To combine `$eq` and `$repeat`: use `exact` and `times` / `value` keys +- To combine `$pattern` and `$neq`: use `dataType`, `exact`, and `matchType` keys + +Refer to the configuration keys from each individual matcher section for the parameters you need. This keeps your composite matchers flexible and prevents redundancy when new matchers or keys are added. + +#### Example + +**Requirement**: Validate that an incoming request field matches the `PaymentInfo` schema partially and can be used for 3 unique values before exhaustion. + +```json +{ + "transient": true, + "http-request": { + "method": "POST", + "path": "/payment", + "body": { + "info": "$match(dataType: PaymentInfo, partial: true, times: 3, value: each)" + } + }, + "http-response": { + "status": 200, + "body": { + "transactionId": "12345" + } + } +} +``` + +### Simulating Downstream Dependency Failures using matchers + +One of the most valuable applications of matchers is **resilience testing**, for example, by utilizing the `delay-in-seconds` feature within Specmatic stubs, you can verify that your API implementation enforces strict timeouts when downstream services become unresponsive or slow. + +**Scenario**: Your API depends on a payment service. You need to verify that your implementation does not wait indefinitely for a slow downstream response, which could lead to thread pool exhaustion or hung connections. Instead, your API should detect the latency, abort the connection early, and return a `429 (Too Many Requests)` or appropriate failure status to the client. + +#### First 2 Requests - Simulated Latency + +The stub introduces a 5-second delay before responding. The goal is to prove that your implementation does not wait the full 5 seconds. +It should ideally timeout earlier (e.g., at 2 seconds) and release the connection. + +```json +{ + "transient": true, + "delay-in-seconds": 5, + "http-request": { + "method": "POST", + "path": "/payments", + "body": { + "amount": "$repeat(times: 2, value: any)" + } + }, + "http-response": { + "status": 200, + "body": { + "status": "delayed_success", + "transactionId": "txn-12345" + }, + } +} + +``` + +#### Requests After Exhaustion - Recovery: + +Once the first matcher exhausts (after 2 requests), the next example takes over. +This simulates the downstream service recovering its performance, allowing your implementation to process requests normally. + +```json +{ + "http-request": { + "method": "POST", + "path": "/payments" + }, + "http-response": { + "status": 200, + "body": { + "status": "success", + "transactionId": "txn-12345" + } + } +} +``` + +**How this works**: +1. **Requests 1-2**: The stub receives the request and delays response by 5 seconds +2. **Verification**: Your API implementation, configured with a shorter timeout (e.g., 2000ms), should trigger a SocketTimeoutException (or equivalent) before the stub responds. +3. **Resource Management**: Instead of holding the connection open, your API catches the timeout, aborts the downstream call, and returns HTTP 429 to the client +4. **Requests 3+**: The first matcher is exhausted. The second example responds immediately with a 200 OK, confirming the system returns to a healthy state once the downstream service recovers + +Specmatic can assist you in testing this scenario during contract testing. In the case of a `429 (Too Many Requests)` response, Specmatic will retry the request, adhering to the specified delay until it either succeeds or the maximum number of retries is reached. For additional details, please refer to the [Smart Resiliency Orchestration](/contract_driven_development/contract_testing.html#smart-resiliency-orchestration) section. + ## SSL / HTTPS Stubbing There are multiple ways to run the Specmatic Stub with SSL. From ef41148d0af1fdfe0f05e933a755eb47eb4d9c3a Mon Sep 17 00:00:00 2001 From: Joel Rosario Date: Thu, 27 Nov 2025 15:10:15 +0530 Subject: [PATCH 2/6] Updated dictionary documentation --- features/dictionary.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/features/dictionary.md b/features/dictionary.md index 719a8efba..fb585a61f 100644 --- a/features/dictionary.md +++ b/features/dictionary.md @@ -23,6 +23,7 @@ redirect_from: * [Examples](#examples) * [Generating the Dictionary](#generating-the-dictionary) * [Understanding the Dictionary](#understanding-the-dictionary) + * [Example Directory Parameter](#example-directory-parameter) * [Dictionary with Contract Testing](#dictionary-with-contract-testing) * [Run the tests](#run-the-tests) * [Generative Tests](#generative-tests) @@ -545,6 +546,14 @@ We have not included the invalid value of `department` in the bad-request exampl the command will include only valid values in the dictionary, allowing it to be executed even with invalid examples. {: .note } +### Example Directory Parameter + +If you have examples in some other existing directory, you could provide the path to that directory using the `--examples-dir` option as shown below: + +```shell +docker run --rm -v "$(pwd)/employees.yaml:/usr/src/app/employees.yaml" -v "$(pwd)/examples:/usr/src/app/examples" -v "$(pwd)/employees_dictionary.yaml:/usr/src/app/employees_dictionary.yaml" specmatic/specmatic-openapi examples dictionary --spec-file employees.yaml --examples-dir ./examples +``` + ## Dictionary with Contract Testing The Dictionary can be utilized in contract testing, allowing Specmatic to use the values defined in the dictionary when generating requests for tests. To illustrate this process, we will use the previous specification and dictionary as an example. From c1f5d774387bdf5b780e93db2243fa79f1be11ca Mon Sep 17 00:00:00 2001 From: Joel Rosario Date: Fri, 28 Nov 2025 17:26:35 +0530 Subject: [PATCH 3/6] First cut improvements to matchers doc --- .../service_virtualization.md | 579 ++++++++---------- 1 file changed, 269 insertions(+), 310 deletions(-) diff --git a/contract_driven_development/service_virtualization.md b/contract_driven_development/service_virtualization.md index 55723c345..306c4da3e 100644 --- a/contract_driven_development/service_virtualization.md +++ b/contract_driven_development/service_virtualization.md @@ -13,103 +13,84 @@ Service Virtualization Mocking -- [Service Virtualization](#service-virtualization) - - [Pre-requisites](#pre-requisites) - - [Inline Examples](#inline-examples) - - [Inline Examples for Responses with No Body](#inline-examples-for-responses-with-no-body) - - [Example Usage](#example-usage) - - [Key Points:](#key-points) - - [Externalizing Example Data](#externalizing-example-data) - - [Handling No Response Body APIs](#handling-no-response-body-apis) - - [Intelligent Service Virtualization - Example cannot go out of sync](#intelligent-service-virtualization---example-cannot-go-out-of-sync) - - [Strict Mode](#strict-mode) - - [Data Type-Based Examples](#data-type-based-examples) - - [Plain Text Request Bodies - Examples With Regular Expressions](#plain-text-request-bodies---examples-with-regular-expressions) - - [Correlated Request And Response Values](#correlated-request-and-response-values) - - [Direct Substitution - Copying Values From Request To Response](#direct-substitution---copying-values-from-request-to-response) - - [Data Lookup](#data-lookup) - - [Partial Examples](#partial-examples) - - [Partial Request Examples](#partial-request-examples) - - [Partial Response Examples](#partial-response-examples) - - [Use Meaningful Response Values From An External Dictionary](#use-meaningful-response-values-from-an-external-dictionary) - - [Delay Simulation](#delay-simulation) - - [Example Specific Delay](#example-specific-delay) - - [Global Delay](#global-delay) - - [Request Matchers](#request-matchers) - - [Why Use Matchers?](#why-use-matchers) - - [Key Benefits](#key-benefits) - - [Equality Matcher: `$eq`](#equality-matcher-eq) - - [Matcher Reference](#matcher-reference) - - [Configuration Keys](#configuration-keys) - - [Example](#example) - - [Inequality Matcher: `$neq`](#inequality-matcher-neq) - - [Matcher Reference](#matcher-reference-1) - - [Configuration Keys](#configuration-keys-1) - - [Example](#example-1) - - [Pattern Matcher: `$pattern`](#pattern-matcher-pattern) - - [Matcher Reference](#matcher-reference-2) - - [Configuration Keys](#configuration-keys-2) - - [Matching Modes](#matching-modes) - - [Example 1: Built-in Type Validation](#example-1-built-in-type-validation) - - [Example 2: Custom Schema with Partial Matching](#example-2-custom-schema-with-partial-matching) - - [Repetition Matcher: `$repeat`](#repetition-matcher-repeat) - - [Matcher Reference](#matcher-reference-3) - - [Configuration Keys](#configuration-keys-3) - - [Counting Strategies](#counting-strategies) - - [Stub Exhaustion Rules](#stub-exhaustion-rules) - - [Example 1: Total Volume Repetition](#example-1-total-volume-repetition) - - [Example 2: Unique Value Repetition](#example-2-unique-value-repetition) - - [Example 3: Combined Repetition Strategies](#example-3-combined-repetition-strategies) - - [Composite Matcher: `$match`](#composite-matcher-match) - - [Matcher Reference](#matcher-reference-4) - - [Using Multiple Matchers Together](#using-multiple-matchers-together) - - [Example](#example-2) - - [Simulating Downstream Dependency Failures using matchers](#simulating-downstream-dependency-failures-using-matchers) - - [First 2 Requests - Simulated Latency](#first-2-requests---simulated-latency) - - [Requests After Exhaustion - Recovery:](#requests-after-exhaustion---recovery) - - [SSL / HTTPS Stubbing](#ssl--https--stubbing) - - [Auto-Generated Cert Store](#auto-generated-cert-store) - - [Bring Your Own Key Store](#bring-your-own-key-store) - - [Dynamic Expectations (a.k.a. Dynamic Stubbing Or Mocking) - Setting Expectations Over Specmatic Http Api](#dynamic-expectations-aka-dynamic-stubbing-or-mocking---setting-expectations-over-specmatic-http-api) - - [Context](#context) - - [Expectations Http Endpoint](#expectations-http-endpoint) - - [Anatomy of a Component / API Test](#anatomy-of-a-component--api-test) - - [Programmatically Starting Stub Server Within Tests](#programmatically-starting-stub-server-within-tests) - - [Transient Expectations (a.k.a. Transient Stubs)](#transient-expectations-aka-transient-stubs) - - [Setting transient expectations](#setting-transient-expectations) - - [Clearing Transient Expectations](#clearing-transient-expectations) - - [Externalised Response Generation](#externalised-response-generation) - - [Hooks](#hooks) - - [Overview](#overview) - - [Use Case: API Gateway Header Transformation](#use-case-api-gateway-header-transformation) - - [Implementation Steps](#implementation-steps) - - [How It Works](#how-it-works) - - [Precedence Across Types Of Examples](#precedence-across-types-of-examples) - - [Checking Health Status Of Stub Server](#checking-health-status-of-stub-server) - - [Example `curl` Request:](#example-curl-request) - - [Specmatic Configuration with Base URL, Host, Port, and Path](#specmatic-configuration-with-base-url-host-port-and-path) - - [Specifying Host](#specifying-host) - - [Specifying Port](#specifying-port) - - [Specifying Base Path](#specifying-base-path) - - [Specifying Combination of Host, Port, and Base Path](#specifying-combination-of-host-port-and-base-path) - - [Specifying Base URL](#specifying-base-url) - - [Customizing the Default Base URL](#customizing-the-default-base-url) - - [Running Specmatic Stubs on Multiple BaseURLs](#running-specmatic-stubs-on-multiple-baseurls) - - [Overview](#overview-1) - - [Directory Structure](#directory-structure) - - [Specmatic Configuration](#specmatic-configuration) - - [API Specifications](#api-specifications) - - [imported\_product.yaml](#imported_productyaml) - - [exported\_product.yaml](#exported_productyaml) - - [Examples](#examples) - - [post\_imported\_product.json](#post_imported_productjson) - - [post\_exported\_product.json](#post_exported_productjson) - - [Run the stub server](#run-the-stub-server) - - [Example Requests](#example-requests) - - [Benefits](#benefits) - - [Disable hot-reload](#disable-hot-reload) - - [Using matching branches in the central contract repo](#using-matching-branches-in-the-central-contract-repo) - - [Sample Java Project](#sample-java-project) +* [Service Virtualization](#service-virtualization) + * [Pre-requisites](#pre-requisites) + * [Inline Examples](#inline-examples) + * [Inline Examples for Responses with No Body](#inline-examples-for-responses-with-no-body) + * [Example Usage](#example-usage) + * [Key Points:](#key-points) + * [Externalizing Example Data](#externalizing-example-data) + * [Handling No Response Body APIs](#handling-no-response-body-apis) + * [Intelligent Service Virtualization - Example cannot go out of sync](#intelligent-service-virtualization---example-cannot-go-out-of-sync) + * [Strict Mode](#strict-mode) + * [Data Type-Based Examples](#data-type-based-examples) + * [Plain Text Request Bodies - Examples With Regular Expressions](#plain-text-request-bodies---examples-with-regular-expressions) + * [Correlated Request And Response Values](#correlated-request-and-response-values) + * [Direct Substitution - Copying Values From Request To Response](#direct-substitution---copying-values-from-request-to-response) + * [Data Lookup](#data-lookup) + * [Partial Examples](#partial-examples) + * [Partial Request Examples](#partial-request-examples) + * [Partial Response Examples](#partial-response-examples) + * [Use Meaningful Response Values From An External Dictionary](#use-meaningful-response-values-from-an-external-dictionary) + * [Delay Simulation](#delay-simulation) + * [Example Specific Delay](#example-specific-delay) + * [Global Delay](#global-delay) + * [Request Matchers](#request-matchers) + * [Why Use Matchers?](#why-use-matchers) + * [Key Benefits](#key-benefits) + * [`$eq`](#eq) + * [`$neq`](#neq) + * [`$match`](#match) + * [Parameter: exact](#parameter-exact) + * [Parameter: datatype](#parameter-datatype) + * [Parameters: value and times](#parameters-value-and-times) + * [Multiple matchers with **value** and **times**](#multiple-matchers-with-value-and-times) + * [Simulating Downstream Dependency Failures using matchers](#simulating-downstream-dependency-failures-using-matchers) + * [First 2 Requests - Simulated Latency](#first-2-requests---simulated-latency) + * [Requests After Exhaustion - Recovery:](#requests-after-exhaustion---recovery) + * [SSL / HTTPS Stubbing](#ssl--https--stubbing) + * [Auto-Generated Cert Store](#auto-generated-cert-store) + * [Bring Your Own Key Store](#bring-your-own-key-store) + * [Dynamic Expectations (a.k.a. Dynamic Stubbing Or Mocking) - Setting Expectations Over Specmatic Http Api](#dynamic-expectations-aka-dynamic-stubbing-or-mocking---setting-expectations-over-specmatic-http-api) + * [Context](#context) + * [Expectations Http Endpoint](#expectations-http-endpoint) + * [Anatomy of a Component / API Test](#anatomy-of-a-component--api-test) + * [Programmatically Starting Stub Server Within Tests](#programmatically-starting-stub-server-within-tests) + * [Transient Expectations (a.k.a. Transient Stubs)](#transient-expectations-aka-transient-stubs) + * [Setting transient expectations](#setting-transient-expectations) + * [Clearing Transient Expectations](#clearing-transient-expectations) + * [Externalised Response Generation](#externalised-response-generation) + * [Hooks](#hooks) + * [Overview](#overview) + * [Use Case: API Gateway Header Transformation](#use-case-api-gateway-header-transformation) + * [Implementation Steps](#implementation-steps) + * [How It Works](#how-it-works) + * [Precedence Across Types Of Examples](#precedence-across-types-of-examples) + * [Checking Health Status Of Stub Server](#checking-health-status-of-stub-server) + * [Example `curl` Request:](#example-curl-request) + * [Specmatic Configuration with Base URL, Host, Port, and Path](#specmatic-configuration-with-base-url-host-port-and-path) + * [Specifying Host](#specifying-host) + * [Specifying Port](#specifying-port) + * [Specifying Base Path](#specifying-base-path) + * [Specifying Combination of Host, Port, and Base Path](#specifying-combination-of-host-port-and-base-path) + * [Specifying Base URL](#specifying-base-url) + * [Customizing the Default Base URL](#customizing-the-default-base-url) + * [Running Specmatic Stubs on Multiple BaseURLs](#running-specmatic-stubs-on-multiple-baseurls) + * [Overview](#overview-1) + * [Directory Structure](#directory-structure) + * [Specmatic Configuration](#specmatic-configuration) + * [API Specifications](#api-specifications) + * [imported_product.yaml](#imported_productyaml) + * [exported_product.yaml](#exported_productyaml) + * [Examples](#examples) + * [post_imported_product.json](#post_imported_productjson) + * [post_exported_product.json](#post_exported_productjson) + * [Run the stub server](#run-the-stub-server) + * [Example Requests](#example-requests) + * [Benefits](#benefits) + * [Disable hot-reload](#disable-hot-reload) + * [Using matching branches in the central contract repo](#using-matching-branches-in-the-central-contract-repo) + * [Sample Java Project](#sample-java-project) ## Pre-requisites @@ -1092,7 +1073,7 @@ Otherwise, the global delay will be applied. Specmatic provides a range of matchers that can be used to compare incoming requests with predefined examples and deliver the appropriate responses -These matchers can match `data types`, `exact values`, and `repetitions`, etc, and Specmatic will only respond with the corresponding response if all the matchers are satisfied +These matchers can match data types, exact values, and repetitions, etc, and Specmatic will only respond with the corresponding response if all the matchers are satisfied {:.note} > It is critical to understand that Specmatic enforces validations in a specific order: @@ -1110,32 +1091,11 @@ Matchers provide fine-grained control over how your stub server responds to inco - **Flexible Request Matching**: Respond differently based on field values, types, or patterns, etc. - **Schema Validation**: Enforce incoming requests to conform to sub-schemas before returning responses -### Equality Matcher: `$eq` +### `$eq` -**Purpose**: Validate exact equality of a field value in incoming requests +Use it to validate exact value of a field value in incoming requests. -#### Matcher Reference - -| Attribute | Value | -| ---------------- | ------------------------------------------------------------------------------- | -| Syntax | `$eq(Value)` | -| Description | Succeeds only if the incoming request value is identical to the specified value | -| Supported Types | Strings, numbers, and other data types | -| Case Sensitivity | Yes | - - -#### Configuration Keys - -| Key | Description | -| ----------- | --------------------------------- | -| `exact` | The exact value to match | -| `matchType` | MatchType defaults to eq (eqauls) | - - -#### Example - -**Requirement**: Confirm that the incoming user's role is strictly `Admin`, -The stub responds with HTTP 200 OK only if the matcher is satisfied. +For example, to ensure that only requests with `role` set to `Admin` receive a successful response, you can use the `$eq` matcher as shown below. ```json { @@ -1153,29 +1113,11 @@ The stub responds with HTTP 200 OK only if the matcher is satisfied. } ``` -### Inequality Matcher: `$neq` - -**Purpose**: Validate that a field value in incoming requests differs from a specified value +### `$neq` -#### Matcher Reference +Use this matcher to validate that a field value in incoming requests is NOT equal to a specified value. -| Attribute | Value | -| ---------------- | ----------------------------------------------------------------------- | -| Syntax | `$neq(Value)` | -| Description | Succeeds if the incoming request value differs from the specified value | -| Supported Types | Strings, numbers, and other data types | -| Case Sensitivity | Yes | - -#### Configuration Keys - -| Key | Description | -| ----------- | --------------------------------------- | -| `exact` | The value to not match | -| `matchType` | Set to `neq` (not equal) for inequality | - -#### Example - -**Requirement**: Confirm that the incoming request status is NOT `Pending`, The stub responds with HTTP 200 OK only if the matcher is satisfied. +For example, to ensure that requests with `status` not equal to `Pending` receive a successful response, you can use the `$neq` matcher as shown below. ```json { @@ -1193,301 +1135,318 @@ The stub responds with HTTP 200 OK only if the matcher is satisfied. } ``` -### Pattern Matcher: `$pattern` - -**Purpose**: Validate that a field's structure in incoming requests matches a specific schema (built-in types or custom schemas). - -#### Matcher Reference - -| Attribute | Value | -| ------------------ | ------------------------------------------------------------------------------------------------------ | -| Syntax | `$pattern(Schema)` or `$pattern(dataType: Schema, partial: boolean)` | -| Description | Validates incoming request data against a schema definition with configurable partial matching support | -| Supported Patterns | Built-in types (`uuid`, `email`, `date`, etc.) and custom schemas from your specification | -| Matching Modes | Strict or Partial | - -#### Configuration Keys +### `$match` -| Key | Description | Values | -| ---------- | -------------------------------------- | --------------------------------------------- | -| `dataType` | The schema name to validate against | Built-in type or custom schema name | -| `partial` | Allow missing fields during validation | `true` (partial) or `false` (strict, default) | +**Purpose**: Use multiple validation rules in a single flexible matcher configuration for incoming requests. -#### Matching Modes +Syntax: `$matcher(exact: value, dataType: datatype, partial: true, times: 2, value: each/any)` -| Mode | Behavior | When to Use | -| ----------- | ---------------------------------------------------------------------------------------- | ----------------------------------------------------- | -| **Strict** | Incoming data must contain all required fields. Missing fields cause validation failure. | When exact schema compliance is required | -| **Partial** | Keys present must match the schema, but missing keys are allowed. | When validating subset of data (e.g., PATCH requests) | +Let's take each parameter one by one. -#### Example 1: Built-in Type Validation +#### Parameter: exact -**Requirement**: Ensure that the incoming payment request contains a valid UUID for the `transactionId`, The stub only responds with `processing` if the matcher is satisfied. +Validates that the field value in incoming requests is exactly equal to the specified value. -```json +``` { "http-request": { "method": "POST", - "path": "/processPayment", + "path": "/greet", "body": { - "amount": 100, - "transactionId": "$pattern(uuid)" + "message": "$match(exact: hello)" } }, "http-response": { "status": 200, "body": { - "status": "processing" + "reply": "Hi there!" } } } ``` -#### Example 2: Custom Schema with Partial Matching +This will match the following request: + +```json +{ + "message": "hello" +} +``` + +It will not match: -**Requirement**: Validate an incoming `PATCH` request where only some contact fields may be updated. -The stub should accept partial updates that match the `ContactInfo` schema structure. +```json +{ + "message": "hi" +} +``` -**API Specification**: +#### Parameter: datatype + +Validates that the field structure in incoming requests matches a specific schema (built-in types or custom schemas). + +Consider the following OpenAPI specification snippet: ```yaml paths: - '/users/{id}': + /order: patch: - summary: Update user details + summary: Place an order requestBody: content: application/json: schema: type: object properties: - contact: - type: object + id: + type: integer + details: + oneOf: + - type: string + - type: integer + responses: + '200': + description: Order Response + content: + application/json: + schema: + type: object + properties: + status: + type: string components: schemas: - ContactInfo: + OrderDetails: type: object required: - - email - - phone - - city + - itemId + - quantity properties: - email: - type: string - format: email - phone: - type: string - city: + itemId: + type: integer + quantity: + type: integer + notes: type: string ``` -**Example File**: +And the following example file: ```json { "http-request": { "method": "PATCH", - "path": "/users/123", + "path": "/order", "body": { - "contact": "$pattern(dataType: ContactInfo, partial: true)" + "id": 10, + "details": "$match(dataType: string)" } }, "http-response": { "status": 200, - "status-text": "Updated" + "body": { + "status": "Order received" + } } } ``` -**How this works**: -1. **Spec Validation**: The PATCH endpoint allows `contact` as a flexible object, so this passes. -2. **Matcher Validation**: The matcher validates that fields in the incoming `contact` object conform to `ContactInfo`, Missing fields are **not** treated as failures because `partial: true` is specified. - -### Repetition Matcher: `$repeat` +Specmatic stub will match this example with the following request: -**Purpose**: Track and limit how many times a specific example response can be returned across multiple incoming requests (stateful matcher), For this to work the example must be marked as `transient` +```json +{ + "id": 10, + "details": "This is a sample order" +} +``` -#### Matcher Reference +The response returned will be: -| Attribute | Value | -| ----------- | ------------------------------------------------------------------------------------- | -| Syntax | `$repeat(times: N, value: Strategy)` | -| Description | Limits the reusability of an example based on incoming request count or unique values | -| Stateful | Yes (tracks across multiple requests) | -| Exhaustion | Example becomes unavailable after matcher conditions are met | +```json +{ + "status": "Order received" +} +``` -#### Configuration Keys +It will not match: -| Key | Description | Values | -| ------- | ---------------------------------------------------------- | --------------------------------------------- | -| `value` | Counting strategy for determining usage | `any` (total count) or `each` (unique values) | -| `times` | Number of times the matcher allows reuse before exhaustion | Integer (e.g., `3`, `5`, `10`) | +```json +{ + "id": 10, + "details": 12345 +} +``` -#### Counting Strategies +Note that although details may be a number in the specification, it must be a string to match this example due to the `$match(dataType: string)` matcher. -| Strategy | Logic | Example | -| -------- | ------------------------------------------------------------------------------------------ | ------------------------------------------ | -| `any` | Every incoming matching request consumes 1 count, irrespective of data content | Three requests → count reaches 3 | -| `each` | Only unique values in incoming requests consume a count; identical values do not increment | Three identical requests → count reaches 1 | -#### Stub Exhaustion Rules +#### Parameters: value and times -An example is considered **exhausted** when: -1. All repetition matchers within the example have met their `times` limit in accordance with their `value` strategy -2. Combination of the repetition matchers has also been exhausted `N` times, where `N` is the maximum of all repetition matchers within the example +**value**: Works with **times** to define the counting strategy for determining usage. -Once exhausted, Specmatic stops returning this example and moves to the next matching example. +When the example has matched requests the defined number of times, it becomes **exhausted** and will not match further requests. In other words, when a request comes in, it will not match this example, even if the datatypes align, when the example has been **exhausted**. -#### Example 1: Total Volume Repetition +These parameters make the example transient, and hence "transient": true must also be set in the example to make it transient. -**Requirement**: The `/status` endpoint should return the same response for the first 3 incoming requests (total volume count), -After 3 requests, the example is exhausted and the next matching example is used. +Consider this example: -```json +``` { "transient": true, "http-request": { - "method": "GET", - "path": "/status", + "method": "POST", + "path": "/echo", "body": { - "requestId": "$repeat(times: 3, value: any)" + "text": "$match(times: 2, value: any)" } }, "http-response": { "status": 200, "body": { - "message": "Service operational" + "echoedText": "$(text)" } } } ``` -**Request Sequence**: -- Request 1 → Returns response (count: 1) -- Request 2 → Returns response (count: 2) -- Request 3 → Returns response (count: 3, **exhausted**) -- Request 4 → Uses next matching example - -#### Example 2: Unique Value Repetition +Specmatic will allow this example to be used two times in total, regardless of the actual value of `text` in incoming requests. After that, the example is considered **exhausted**. An **exhausted** example will match no further requests. -**Requirement**: The `/user/{userId}` endpoint should return the same response once per unique `userId` in incoming requests, -After 5 unique userIds are seen, the example is exhausted. +For example, if Specmatic stub first receives the following request: ```json { - "transient": true, - "http-request": { - "method": "GET", - "path": "/user/(number)", - "body": { - "userId": "$repeat(times: 5, value: each)" - } - }, - "http-response": { - "status": 200, - "body": { - "name": "John Doe" - } - } + "text": "hello" } ``` -**Request Sequence**: -- Request 1 (userId: 1) → Returns response (unique count: 1) -- Request 2 (userId: 1) → Returns response (unique count: 1, no increment) -- Request 3 (userId: 2) → Returns response (unique count: 2) -- Request 4 (userId: 2) → Returns response (unique count: 2, no increment) -- Request 5 (userId: 3) → Returns response (unique count: 3) -- Request 6 (userId: 4) → Returns response (unique count: 4) -- Request 7 (userId: 5) → Returns response (unique count: 5, **exhausted**) -- Request 8 (userId: 6) → Uses next matching example - -#### Example 3: Combined Repetition Strategies +And then receives the following request: -**Requirement**: The `/validate` endpoint contains two repeating fields with different constraints: -1. `category`: Should match up to 2 unique values (value: each) -2. `requestId`: Should match up to 4 total requests (value: any) +```json +{ + "text": "world" +} +``` -**Exhaustion Rule**: The example is considered exhausted when the request count reaches 4 (the maximum of 2 and 4). Until that limit is reached, the stub remains active provided incoming requests satisfy both local matchers and the overall exhaustion rule. Note that individual matchers may reach their limits earlier than the overall exhaustion point. +The matcher has been exercised two times now, so the example is exhausted. Because it it exhausted, it will not match the following request, even though the datatypes align with the example. ```json +{ + "text": "hi there" +} +``` + +But now look at this example: + +``` { "transient": true, "http-request": { "method": "POST", - "path": "/validate", + "path": "/echo", "body": { - "category": "$repeat(times: 2, value: each)", - "requestId": "$repeat(times: 4, value: any)" + "text": "$match(times: 2, value: each)" } }, "http-response": { - "status": 202, + "status": 200, "body": { - "status": "Accepted" + "echoedText": "$(text)" } } } ``` -**Request Sequence**: -- Request 1 (`category`: "A", `requestId`: "101") -- → Returns 202 (category count: 1/2, requestId count: 1/4, **Active**) -- Request 2 (`category`: "B", `requestId`: "102") -- → Returns 202 (category count: 2/2, requestId count: 2/4, **Active**) -- Request 3 (`category`: "C", `requestId`: "103") -- → Returns 202 (category count: 2/2 - *Local Limit Met*, requestId count: 3/4, **Active**) -- Request 4 (`category`: "D", `requestId`: "104") -- → Returns 202 (category count: 2/2 - *Local Limit Met*, requestId count: 4/4, **Exhausted**) -- Request 5 (`category`: "E", `requestId`: "105") → Uses next matching example (first example is fully exhausted) - -### Composite Matcher: `$match` +Specmatic will allow this example to be used two times for each unique value of `text` in incoming requests. -**Purpose**: Combine multiple validation rules into a single flexible matcher configuration for incoming requests. +For example, if Specmatic stub first receives the following request: -#### Matcher Reference +```json +{ + "text": "hello" +} +``` -| Attribute | Value | -| ---------------------- | --------------------------------------------------------------------------------------------------------- | -| Syntax | `$match(key1: value1, key2: value2, ...)` | -| Description | Combines equality, type validation, partial matching, and/or repetition constraints for incoming requests | -| Supported Combinations | Any compatible combination of configuration keys | -| Flexibility | Highly flexible for complex scenarios | +And then receives the following request: -#### Using Multiple Matchers Together +```json +{ + "text": "hello" +} +``` -The `$match` composite matcher allows you to combine validation capabilities from any of the individual matchers (`$eq`, `$neq`, `$pattern`, `$repeat`) within a single field. Simply include the configuration keys you need from the matchers you want to use. +The matcher has been exercised two times for the unique value "hello", so the example is exhausted for that value. Because it it exhausted for "hello", it will not match the same request a third time. -For example: -- To combine `$pattern` and `$repeat`: use `dataType` and `times` / `value` keys -- To combine `$eq` and `$repeat`: use `exact` and `times` / `value` keys -- To combine `$pattern` and `$neq`: use `dataType`, `exact`, and `matchType` keys +However, it will match the following request, because "world" is a new unique value. -Refer to the configuration keys from each individual matcher section for the parameters you need. This keeps your composite matchers flexible and prevents redundancy when new matchers or keys are added. +```json +{ + "text": "world" +} +``` -#### Example +#### Multiple matchers with **value** and **times** -**Requirement**: Validate that an incoming request field matches the `PaymentInfo` schema partially and can be used for 3 unique values before exhaustion. +Consider the following example: -```json +``` { "transient": true, "http-request": { "method": "POST", - "path": "/payment", + "path": "/echo", "body": { - "info": "$match(dataType: PaymentInfo, partial: true, times: 3, value: each)" + "name": "$match(dataType: string, times: 1, value: any)", + "type": "$match(dataType: string, times: 2, value: any)" } }, "http-response": { "status": 200, "body": { - "transactionId": "12345" + "echoedText": "$(text)" } } } ``` +Now consider the following sequence of requests: + +- Request 1: + +```json +{ + "name": "Alice", + "type": "greeting" +} +``` + +After matching and responding with this matcher, the matcher states are: +- name count: 1/1 - **Exhausted** +- type count: 1/2 - **Active** + +- Request 2: + +```json +{ + "name": "Bob", + "type": "greeting" +} +``` + +After matching and responding with this matcher, the matcher states are: +- name count: 1/1 - **Exhausted** +- type count: 2/2 - **Exhausted** + +- Request 3: + +```json +{ + "name": "Charlie", + "type": "farewell" +} +``` + +At this point, both matchers are exhausted, so this request will not match this example. + ### Simulating Downstream Dependency Failures using matchers One of the most valuable applications of matchers is **resilience testing**, for example, by utilizing the `delay-in-seconds` feature within Specmatic stubs, you can verify that your API implementation enforces strict timeouts when downstream services become unresponsive or slow. @@ -1544,11 +1503,11 @@ This simulates the downstream service recovering its performance, allowing your **How this works**: 1. **Requests 1-2**: The stub receives the request and delays response by 5 seconds -2. **Verification**: Your API implementation, configured with a shorter timeout (e.g., 2000ms), should trigger a SocketTimeoutException (or equivalent) before the stub responds. +2. **Verification**: Your API implementation, configured with a shorter timeout (e.g., 2000ms), should throw a TimeoutException or equivalent before the stub responds. 3. **Resource Management**: Instead of holding the connection open, your API catches the timeout, aborts the downstream call, and returns HTTP 429 to the client -4. **Requests 3+**: The first matcher is exhausted. The second example responds immediately with a 200 OK, confirming the system returns to a healthy state once the downstream service recovers +4. **Requests 3+**: The first matcher is exhausted. So when the consumer retries, the second example responds immediately with a 200 OK, thus showing how the system returns to a healthy state once the downstream service recovers -Specmatic can assist you in testing this scenario during contract testing. In the case of a `429 (Too Many Requests)` response, Specmatic will retry the request, adhering to the specified delay until it either succeeds or the maximum number of retries is reached. For additional details, please refer to the [Smart Resiliency Orchestration](/contract_driven_development/contract_testing.html#smart-resiliency-orchestration) section. +Specmatic can assist you in testing this scenario during contract testing. In the case of a `429 (Too Many Requests)` response, Specmatic will retry the request, adhering to the delay specified in the `Retry-After` header. For additional details, please refer to the [Smart Resiliency Orchestration](/contract_driven_development/contract_testing.html#smart-resiliency-orchestration) section. ## SSL / HTTPS Stubbing From 6530c813fc9f89663cb6c0adad3615b48325835b Mon Sep 17 00:00:00 2001 From: Joel Rosario Date: Sun, 30 Nov 2025 12:18:59 +0000 Subject: [PATCH 4/6] Updated documentation on matchers to use-case-based --- .../service_virtualization.md | 790 ++++++++---------- 1 file changed, 349 insertions(+), 441 deletions(-) diff --git a/contract_driven_development/service_virtualization.md b/contract_driven_development/service_virtualization.md index 306c4da3e..5059e058c 100644 --- a/contract_driven_development/service_virtualization.md +++ b/contract_driven_development/service_virtualization.md @@ -35,19 +35,6 @@ Service Virtualization * [Delay Simulation](#delay-simulation) * [Example Specific Delay](#example-specific-delay) * [Global Delay](#global-delay) - * [Request Matchers](#request-matchers) - * [Why Use Matchers?](#why-use-matchers) - * [Key Benefits](#key-benefits) - * [`$eq`](#eq) - * [`$neq`](#neq) - * [`$match`](#match) - * [Parameter: exact](#parameter-exact) - * [Parameter: datatype](#parameter-datatype) - * [Parameters: value and times](#parameters-value-and-times) - * [Multiple matchers with **value** and **times**](#multiple-matchers-with-value-and-times) - * [Simulating Downstream Dependency Failures using matchers](#simulating-downstream-dependency-failures-using-matchers) - * [First 2 Requests - Simulated Latency](#first-2-requests---simulated-latency) - * [Requests After Exhaustion - Recovery:](#requests-after-exhaustion---recovery) * [SSL / HTTPS Stubbing](#ssl--https--stubbing) * [Auto-Generated Cert Store](#auto-generated-cert-store) * [Bring Your Own Key Store](#bring-your-own-key-store) @@ -59,6 +46,19 @@ Service Virtualization * [Transient Expectations (a.k.a. Transient Stubs)](#transient-expectations-aka-transient-stubs) * [Setting transient expectations](#setting-transient-expectations) * [Clearing Transient Expectations](#clearing-transient-expectations) + * [Request Matchers](#request-matchers) + * [Matcher: `$neq`](#matcher-neq) + * [Matcher: `$match`](#matcher-match) + * [Parameter: exact](#parameter-exact) + * [Parameter: datatype](#parameter-datatype) + * [Parameters: `value` and `times`](#parameters-value-and-times) + * [Match each unique value a specified number of times](#match-each-unique-value-a-specified-number-of-times) + * [Match any value a specified number of times](#match-any-value-a-specified-number-of-times) + * [Multiple matchers with **value** and **times**](#multiple-matchers-with-value-and-times) + * [Recap of `value` and `times`](#recap-of-value-and-times) + * [Simulating Downstream Dependency Failures using matchers](#simulating-downstream-dependency-failures-using-matchers) + * [First 2 Requests - Simulated Latency](#first-2-requests---simulated-latency) + * [Requests After Exhaustion - Recovery](#requests-after-exhaustion---recovery) * [Externalised Response Generation](#externalised-response-generation) * [Hooks](#hooks) * [Overview](#overview) @@ -1067,57 +1067,250 @@ stub: {% endtabs %} **Note:** If the delay is specified in the example file, it will be used to simulate response times for that specific example. + Otherwise, the global delay will be applied. -## Request Matchers +## SSL / HTTPS Stubbing + +There are multiple ways to run the Specmatic Stub with SSL. + +### Auto-Generated Cert Store + +This is the quickest approach. +{% tabs test %} +{% tab test java %} +```shell +java -jar specmatic.jar stub --httpsKeyStoreDir= --port=443 product-api.yaml +``` +{% endtab %} +{% tab test npm %} +```shell +npx specmatic stub --httpsKeyStoreDir= --port=443 product-api.yaml +``` +{% endtab %} +{% tab test docker %} +```shell +docker run -p 443:443 -v "${PWD}/product-api.yaml:/usr/src/app/product-api.yaml" -v "${PWD}/:/usr/src/app/" specmatic/specmatic stub --httpsKeyStoreDir= --port=443 product-api.yaml +``` +{% endtab %} +{% endtabs %} + +This will create a `specmatic.jks` file in the dir that you mentioned above, and you can now access the stub over https. + +### Bring Your Own Key Store + +If you already have a keystore and self-signed certificate you can pass it to Specmatic through below command options. + +```shell +% specmatic stub --help +... + --httpsKeyStore= + Run the proxy on https using a key in this store + --httpsKeyStorePassword= + Run the proxy on https, password for pre-existing + key store + --httpsPassword= + Key password if any +``` + +## Dynamic Expectations (a.k.a. Dynamic Stubbing Or Mocking) - Setting Expectations Over Specmatic Http Api + +### Context +It is not always possible to know ahead of time what expectation data needs to be setup. Example: Consider below scenario +* Let us say our system under test needs to look up the SKU value from another service (over which we do not have control or cannot mock) before creating products +* In this scenario we will be unable to create the expectation json file with any specific value of SKU since we will only know this at test runtime / dynamically + +**Dynamic Stubs** are helpful in such scenarios which involve a **work flow** with multiple steps where the input of a step depends on output of its previous step. + +### Expectations Http Endpoint + +Specmatic stub server can accept expectation json through below http endpoint. + +```shell +http://localhost:9000/_specmatic/expectations +``` + +Please see postman collection for reference. + +Specmatic will verify these expectations against the OpenAPI Specifications and will only return a 2xx response if they are as per API Specifications. Specmatic returns 4xx response if the expectation json is not as per the OpenAPI Specifications. + +### Anatomy of a Component / API Test + +Anatomy of a Component / API Test + +Please see this [video](https://youtu.be/U5Agz-mvYIU?t=998) for reference. + +The above image shows how Specmatic Smart Mocking fits into your Component Test. A good component test isolates the system / component under test from its dependencies. Here Specmatic is emulating the dependencies of the mobile application thereby isolating it. + +**API Tests are just Component Tests where the System Under Test is a Service / API**. Here is an [example](https://github.com/specmatic/specmatic-order-bff-java/blob/main/src/test/kotlin/com/component/orders/ApiTests.kt) of how you can leverage Specmatic dynamic mocking in an API Test. Below are the pieces involved. +* **System Under Test** - [Find Available Products Service](https://github.com/specmatic/specmatic-order-bff-java/blob/main/src/main/kotlin/com/component/orders/controllers/Products.kt) - Invokes products API to get all products and filters out products where inventory is zero. +* **Dependency** - Products API mocked by Specmatic. Specmatic is set up to leverage [OpenAPI Specification of Products API](https://github.com/specmatic/specmatic-order-contracts/blob/main/io/specmatic/examples/store/openapi/api_order_v3.yaml) in the [central contract repo](https://github.com/specmatic/specmatic-order-contracts) through [specmatic.yaml](https://github.com/specmatic/specmatic-order-bff-java/blob/main/src/test/resources/specmatic.yaml) configuration. + +Let us analyse each phase of this API test. +* **Arrange** - In this step, we set up Specmatic stub server with expectation json through Specmatic http endpoint to emulate the Products API to return expected response. We also verify that Specmatic has accepted this expectation data by asserting that the response code is 2xx. This confirms that are our expectation data is in line with the OpenAPI Specification of Products OpenAPI Specification. +* **Act** - Here the test invokes System Under Test to exercise the functionality we need to test. This inturn results in the System Under Test invoking its dependency (Products Service) which is being emulated by Specmatic. Specmatic returns the response we have set up in the previous step to the System Under Test. System Under Test processes this data and responds to API Test. +* **Assert** - We now verify the response from System Under Test to ascertain if it has returned the correct response. + +## Programmatically Starting Stub Server Within Tests + +{% tabs virtualization %} +{% tab virtualization java %} +If your tests are written in a JVM based language, you can start and stop the stub server within your tests programmatically. + +Add `specmatic-core` jar dependency with scope set to test since this need not be shipped as part of your production deliverable. + +``` + + io.specmatic + specmatic-core + {{ site.specmatic-core-version }} + test + +``` + +Now you can import the utility to create the stub server. Below code snippets are in Kotlin. However, the overall concept is the same across all JVM languages such as Clojure, Scala or plain Java. + +```kotlin +import io.specmatic.stub.createStub +``` + +This utility can now be used in your test ```setup``` / ```beforeAll``` method to start the stub server. Specmatic automatically looks for your [Specmatic configuration](/references/configuration.html) file in project root directory / classpath to locate your API Specification files that need to run as part of the stub server. + +```kotlin +@BeforeAll +@JvmStatic +fun setUp() { + stub = createStub() +} +``` + +We can now programmatically set [dynamic expectations](#dynamic-expectations-aka-dynamic-stubbing-or-mocking---setting-expectations-over-specmatic-http-api) on the ```stub``` with the ```setExpectation()``` method where `````` is in the same format as [seen here](#externalizing-example-data) + +```java +stub.setExpectation(expectationJson); +``` + +If you have several such JSON expectation files that you would like to set up at once, you can pass a list of files or dir containing these expectation JSON files while creating the stub. + +```kotlin +httpStub = createStub(listOf("./src/test/resources")) +``` + +The above `createStub()` function creates your Specmatic HTTP stub with default host, port, etc. Below is an example with all values being passed in + +```kotlin +@BeforeAll +@JvmStatic +fun setUp() { + stub = createStub(listOf("./src/test/resources"), "localhost", 8090, strict = true) +} +``` + +The last parameter (`strict = true`), enables **strict** mode where Specmatic HTTP Stub will only respond to requests where expectations have been set. For any other requests, `400 Bad Request` is returned. + +And subsequently once your tests are done, you can shut down the stub server as part of your ```teardown``` / ```afterAll``` method. + +```kotlin +@AfterAll +@JvmStatic +fun tearDown() { + service?.stop() + stub.close() +} +``` + +Here is a complete example of Specmatic Contract Tests that leverages the above technique. + +[Kotlin Example](https://github.com/specmatic/specmatic-order-bff-java/blob/main/src/test/kotlin/com/component/orders/contract/ContractTests.kt) + +Please note that this is only a utility for the purpose of convenience in Java projects. Other programming languages can simply run the Specmatic standalone executable just as easily. If you do happen to write a thin wrapper and would like to contribute the same to the project, please refer to our [contribution guidelines](https://github.com/specmatic/specmatic/blob/main/CONTRIBUTING.md). -Specmatic provides a range of matchers that can be used to compare incoming requests with predefined examples and deliver the appropriate responses +{% endtab %} +{% tab virtualization python %} +If your tests are written in Python, you can start and stop the stub server within your tests programmatically. + +1. **Install the Specmatic Python library**: Use pip, a package installer for Python, to install the Specmatic library. + + ```bash + pip install specmatic + ``` -These matchers can match data types, exact values, and repetitions, etc, and Specmatic will only respond with the corresponding response if all the matchers are satisfied +2. **Run Tests with a Stub**: If you want to run the tests with a stub, you can do so like this: -{:.note} -> It is critical to understand that Specmatic enforces validations in a specific order: -> - **OpenAPI Spec Validation**: First, the request is validated against your OpenAPI specification. If a field is marked as required in the spec, it must be present and valid -> - **Matcher Validation**: The custom matcher (like `$pattern` or `$match`) only execute after the request has satisfied the base contract + ```python + import os + from specmatic.core.specmatic import Specmatic + from your_project import app + PROJECT_ROOT_DIR = os.path.dirname(os.path.abspath(__file__)) + app_host = "127.0.0.1" + app_port = 5000 + stub_host = "127.0.0.1" + stub_port = 5000 + expectation_json_file = PROJECT_ROOT_DIR + '/test/data/expectation.json' -### Why Use Matchers? + Specmatic() \ + .with_project_root(PROJECT_ROOT_DIR) \ + .with_stub(stub_host, stub_port, [expectation_json_file]) \ + .with_wsgi_app(app, app_host, app_port) \ + .test(TestContract) \ + .run() + ``` -Matchers provide fine-grained control over how your stub server responds to incoming requests. Instead of relying solely on static examples, matchers enable dynamic, stateful, conditional response selection based on request content. + In this example, we are passing an instance of wsgi app like flask. `stub_host`, `stub_port`, and `expectation_json_file` are the host and port for the stub server, and the path to a JSON file containing expectations for the stub, respectively. Replace `app` with your Flask application object. -#### Key Benefits + Here are complete example of [Specmatic stub server](https://github.com/specmatic/specmatic-order-bff-python/blob/main/test/test_contract.py) usage in Python. +{% endtab %} +{% endtabs %} -- **Stateful Behavior**: Track request counts and simulate exhaustion for advanced scenarios -- **Resilience**: Simulate downstream service behaviors including failures, delays, and edge cases -- **Flexible Request Matching**: Respond differently based on field values, types, or patterns, etc. -- **Schema Validation**: Enforce incoming requests to conform to sub-schemas before returning responses +## Transient Expectations (a.k.a. Transient Stubs) -### `$eq` +A transient mock becomes unavailable immediately after it has been exercised. -Use it to validate exact value of a field value in incoming requests. +### Setting transient expectations -For example, to ensure that only requests with `role` set to `Admin` receive a successful response, you can use the `$eq` matcher as shown below. +For example, create this stub file for products-api.yaml contract: ```json { + "http-stub-id": "123", "http-request": { - "method": "POST", - "path": "/verifyUser", - "body": { - "role": "$eq(Admin)" - } + "method": "GET", + "path": "/storestatus", }, "http-response": { "status": 200, - "status-text": "OK" + "body": "open" } } ``` -### `$neq` +Make an HTTP request to http://localhost:9000/storestatus with the GET method. The response says "open". + +Now try the same request again. The response is a randomized string. + +This is useful particularly when there are no distinguishing features of the request like in the above example, and we need to simulate a succession of calls to that API giving different responses. + +### Clearing Transient Expectations + +If the test fails and you need to start a new run of the test, you may need to clear all the transient mocks so that the two tests do not step on each other's toes. + +To do that, make an API call to the path /_specmatic/http-stub/ with the DELETE verb. + +To clear the transient mock in the above example, you would call http://localhost:9000/_specmatic/http-stub/123 with the DELETE verb. + +## Request Matchers + +Specmatic provides powerful matchers that offer dynamic, fine-grained control over how your stub server responds to incoming requests, enabling you to simulate complex and evolving scenarios. + +{: .note} Requests are validated against the specification first, before any matchers in examples are evaluated. -Use this matcher to validate that a field value in incoming requests is NOT equal to a specified value. +### Matcher: `$neq` -For example, to ensure that requests with `status` not equal to `Pending` receive a successful response, you can use the `$neq` matcher as shown below. +**Syntax:** `$neq()` + +Use this matcher to ensure that a field value in incoming requests is **not equal** to a specific value. + +For example, the following example matches only requests where `status` is not equal to `Pending`: ```json { @@ -1135,25 +1328,24 @@ For example, to ensure that requests with `status` not equal to `Pending` receiv } ``` -### `$match` +### Matcher: `$match` -**Purpose**: Use multiple validation rules in a single flexible matcher configuration for incoming requests. +**Syntax:** `$match(exact: , dataType: , times: , value: each|any)` -Syntax: `$matcher(exact: value, dataType: datatype, partial: true, times: 2, value: each/any)` +This matcher provides flexible options to control how request fields match example data and how many times a particular example can be used. -Let's take each parameter one by one. +#### Match an exact value a fixed number of times -#### Parameter: exact +Consider the following **transient** example: -Validates that the field value in incoming requests is exactly equal to the specified value. - -``` +```json { + "transient": true, "http-request": { "method": "POST", "path": "/greet", "body": { - "message": "$match(exact: hello)" + "message": "$match(exact: hello, times: 2)" } }, "http-response": { @@ -1165,7 +1357,11 @@ Validates that the field value in incoming requests is exactly equal to the spec } ``` -This will match the following request: +This matcher will respond to two requests where `message` is exactly `hello`. After two successful matches, the matcher becomes **exhausted**, and subsequent requests with `{"message": "hello"}` will no longer match this example. + +**Example:** + +Request 1: ```json { @@ -1173,131 +1369,75 @@ This will match the following request: } ``` -It will not match: +Request 2: ```json { - "message": "hi" + "message": "hello" } ``` -#### Parameter: datatype +After these two requests, the matcher becomes **exhausted**, and this transient example will not match further requests such as: -Validates that the field structure in incoming requests matches a specific schema (built-in types or custom schemas). +Request 3: -Consider the following OpenAPI specification snippet: +```json +{ + "message": "hello" +} +``` -```yaml -paths: - /order: - patch: - summary: Place an order - requestBody: - content: - application/json: - schema: - type: object - properties: - id: - type: integer - details: - oneOf: - - type: string - - type: integer - responses: - '200': - description: Order Response - content: - application/json: - schema: - type: object - properties: - status: - type: string -components: - schemas: - OrderDetails: - type: object - required: - - itemId - - quantity - properties: - itemId: - type: integer - quantity: - type: integer - notes: - type: string -``` +If there are multiple matchers in an example, the example remains active until **all** of its matchers are exhausted. + +##### Each unique value matched a fixed number of times -And the following example file: +Consider the following **transient** example: ```json { + "transient": true, "http-request": { - "method": "PATCH", - "path": "/order", + "method": "POST", + "path": "/echo", "body": { - "id": 10, - "details": "$match(dataType: string)" + "text": "$match(dataType: string, value: each, times: 2)" } }, "http-response": { "status": 200, "body": { - "status": "Order received" + "echoedText": "$(text)" } } } ``` -Specmatic stub will match this example with the following request: +Each unique value for `text` can be matched twice. For example: -```json -{ - "id": 10, - "details": "This is a sample order" -} -``` - -The response returned will be: - -```json -{ - "status": "Order received" -} -``` - -It will not match: +**Request payload 1:** `{ "text": "hello" }` -```json -{ - "id": 10, - "details": 12345 -} -``` +**Request payload 2:** `{ "text": "hello" }` -Note that although details may be a number in the specification, it must be a string to match this example due to the `$match(dataType: string)` matcher. +After these two requests, the matcher for `text` with value `hello` becomes **exhausted**. +**Request payload 3:** `{ "text": "world" }` -#### Parameters: value and times +The value "world" has never been seen before, and hence this matcher will be active for "world", and will match it twice before getting exhausted. -**value**: Works with **times** to define the counting strategy for determining usage. +If there are multiple matchers in an example, the example remains active until **all** of its matchers are exhausted. -When the example has matched requests the defined number of times, it becomes **exhausted** and will not match further requests. In other words, when a request comes in, it will not match this example, even if the datatypes align, when the example has been **exhausted**. +##### Any value matched a fixed number of times -These parameters make the example transient, and hence "transient": true must also be set in the example to make it transient. +Consider this **transient** example: -Consider this example: - -``` +```json { "transient": true, "http-request": { "method": "POST", "path": "/echo", "body": { - "text": "$match(times: 2, value: any)" + "text": "$match(dataType: string, value: any, times: 2)" } }, "http-response": { @@ -1309,146 +1449,142 @@ Consider this example: } ``` -Specmatic will allow this example to be used two times in total, regardless of the actual value of `text` in incoming requests. After that, the example is considered **exhausted**. An **exhausted** example will match no further requests. - -For example, if Specmatic stub first receives the following request: - -```json -{ - "text": "hello" -} -``` +Here, the matcher can match any value twice, regardless of what it is. After two matches, it becomes **exhausted**, and the example will no longer be used. -And then receives the following request: +#### Multiple `value: each` matchers working together -```json -{ - "text": "world" -} -``` +In a previous section we looked at an example with a single `value: each` matcher. But now, let's see what happens when there are multiple `value: each` matchers work. -The matcher has been exercised two times now, so the example is exhausted. Because it it exhausted, it will not match the following request, even though the datatypes align with the example. +Consider this **transient** example where both `id` and `details` use `value: each` with `times: 1`: ```json -{ - "text": "hi there" -} -``` - -But now look at this example: - -``` { "transient": true, "http-request": { - "method": "POST", - "path": "/echo", + "method": "PATCH", + "path": "/order/10", "body": { - "text": "$match(times: 2, value: each)" + "id": "$match(dataType: integer, times: 1, value: each)", + "details": "$match(dataType: string, times: 1, value: each)" } }, "http-response": { "status": 200, "body": { - "echoedText": "$(text)" + "status": "Order updated" } } } ``` -Specmatic will allow this example to be used two times for each unique value of `text` in incoming requests. +When there are multiple `value: each` matchers in an example, Specmatic tracks the usage of each unique combination of matcher values. + +Let's walk through a sequence of requests. + +**Initial state:** + +* No `(id, details)` pairs have been used. +* `id` matcher: all values are **Active**. +* `details` matcher: all values are **Active**. -For example, if Specmatic stub first receives the following request: +**Request 1** ```json { - "text": "hello" + "id": 10, + "details": "packed" } ``` -And then receives the following request: +This is the first time the stub sees `id: 10` together with `details: "packed"`, so it matches. + +**Matcher state after Request 1:** + +* Pair `id: 10`, `details: "packed"`: used 1/1 (**Exhausted** for this pair) + +**Request 2** ```json { - "text": "hello" + "id": 10, + "details": "packed" } ``` -The matcher has been exercised two times for the unique value "hello", so the example is exhausted for that value. Because it it exhausted for "hello", it will not match the same request a third time. +This is the exact same pair as Request 1. -However, it will match the following request, because "world" is a new unique value. +**Matcher state after Request 2:** + +* Pair `id: 10`, `details: "packed"` is already 1/1 (**Exhausted**) → this request will **not** match this example. + +**Request 3** ```json { - "text": "world" + "id": 10, + "details": "shipped" } ``` -#### Multiple matchers with **value** and **times** +This is a new pair: `id: 10` with `details: "shipped"`. -Consider the following example: +**Matcher state after Request 3:** -``` -{ - "transient": true, - "http-request": { - "method": "POST", - "path": "/echo", - "body": { - "name": "$match(dataType: string, times: 1, value: any)", - "type": "$match(dataType: string, times: 2, value: any)" - } - }, - "http-response": { - "status": 200, - "body": { - "echoedText": "$(text)" - } - } -} -``` +* Pair `id: 10`, `details: "packed"`: **Exhausted**. +* Pair `id: 10`, `details: "shipped"`: **Exhausted**. -Now consider the following sequence of requests: +Even though `id = 10` was seen before, this request works because this exact combination (`10` + `"shipped"`) had not been used previously. -- Request 1: +**Request 4** ```json { - "name": "Alice", - "type": "greeting" + "id": 10, + "details": "shipped" } ``` -After matching and responding with this matcher, the matcher states are: -- name count: 1/1 - **Exhausted** -- type count: 1/2 - **Active** +This reuses the pair from Request 3. -- Request 2: +**Matcher state after Request 4:** + +* Pair `id: 10`, `details: "shipped"` is already 1/1 (**Exhausted**) → this request does **not** match this example. + +**Request 5** ```json { - "name": "Bob", - "type": "greeting" + "id": 20, + "details": "packed" } ``` -After matching and responding with this matcher, the matcher states are: -- name count: 1/1 - **Exhausted** -- type count: 2/2 - **Exhausted** +Now we have a new pair: `id: 20` with `details: "packed"`. -- Request 3: +**Matcher state after Request 5:** -```json -{ - "name": "Charlie", - "type": "farewell" -} -``` +* Pair `id: 10`, `details: "packed"`: **Exhausted**. +* Pair `id: 10`, `details: "shipped"`: **Exhausted**. +* Pair `id: 20`, `details: "packed"`: used 1/1 (**Exhausted** for this pair). + +This request matches because, although `"packed"` was seen earlier with `id = 10`, the combination `id: 20`, `details: "packed"` is new. + +In summary: + +* The stub accepts each unique `id` + `details` combination exactly once. +* Reusing the same pair again will not match the example. +* Reusing either `id` or `details` with a new partner is allowed until that new combination has also been used once. + +**Key takeaway:** -At this point, both matchers are exhausted, so this request will not match this example. +* Each matcher (`name` and `type`) tracks its own count independently. +* Once a matcher reaches its `times` limit, it moves from **Active** to **Exhausted** and stops matching. +* When all matchers in a transient example are exhausted, the example itself is considered exhausted and will no longer be used to match incoming request payloads. ### Simulating Downstream Dependency Failures using matchers +Let's bring this together with a practical example. + One of the most valuable applications of matchers is **resilience testing**, for example, by utilizing the `delay-in-seconds` feature within Specmatic stubs, you can verify that your API implementation enforces strict timeouts when downstream services become unresponsive or slow. **Scenario**: Your API depends on a payment service. You need to verify that your implementation does not wait indefinitely for a slow downstream response, which could lead to thread pool exhaustion or hung connections. Instead, your API should detect the latency, abort the connection early, and return a `429 (Too Many Requests)` or appropriate failure status to the client. @@ -1480,7 +1616,7 @@ It should ideally timeout earlier (e.g., at 2 seconds) and release the connectio ``` -#### Requests After Exhaustion - Recovery: +#### Requests After Exhaustion - Recovery Once the first matcher exhausts (after 2 requests), the next example takes over. This simulates the downstream service recovering its performance, allowing your implementation to process requests normally. @@ -1509,234 +1645,6 @@ This simulates the downstream service recovering its performance, allowing your Specmatic can assist you in testing this scenario during contract testing. In the case of a `429 (Too Many Requests)` response, Specmatic will retry the request, adhering to the delay specified in the `Retry-After` header. For additional details, please refer to the [Smart Resiliency Orchestration](/contract_driven_development/contract_testing.html#smart-resiliency-orchestration) section. -## SSL / HTTPS Stubbing - -There are multiple ways to run the Specmatic Stub with SSL. - -### Auto-Generated Cert Store - -This is the quickest approach. -{% tabs test %} -{% tab test java %} -```shell -java -jar specmatic.jar stub --httpsKeyStoreDir= --port=443 product-api.yaml -``` -{% endtab %} -{% tab test npm %} -```shell -npx specmatic stub --httpsKeyStoreDir= --port=443 product-api.yaml -``` -{% endtab %} -{% tab test docker %} -```shell -docker run -p 443:443 -v "${PWD}/product-api.yaml:/usr/src/app/product-api.yaml" -v "${PWD}/:/usr/src/app/" specmatic/specmatic stub --httpsKeyStoreDir= --port=443 product-api.yaml -``` -{% endtab %} -{% endtabs %} - -This will create a `specmatic.jks` file in the dir that you mentioned above, and you can now access the stub over https. - -### Bring Your Own Key Store - -If you already have a keystore and self-signed certificate you can pass it to Specmatic through below command options. - -```shell -% specmatic stub --help -... - --httpsKeyStore= - Run the proxy on https using a key in this store - --httpsKeyStorePassword= - Run the proxy on https, password for pre-existing - key store - --httpsPassword= - Key password if any -``` - -## Dynamic Expectations (a.k.a. Dynamic Stubbing Or Mocking) - Setting Expectations Over Specmatic Http Api - -### Context -It is not always possible to know ahead of time what expectation data needs to be setup. Example: Consider below scenario -* Let us say our system under test needs to look up the SKU value from another service (over which we do not have control or cannot mock) before creating products -* In this scenario we will be unable to create the expectation json file with any specific value of SKU since we will only know this at test runtime / dynamically - -**Dynamic Stubs** are helpful in such scenarios which involve a **work flow** with multiple steps where the input of a step depends on output of its previous step. - -### Expectations Http Endpoint - -Specmatic stub server can accept expectation json through below http endpoint. - -```shell -http://localhost:9000/_specmatic/expectations -``` - -Please see postman collection for reference. - -Specmatic will verify these expectations against the OpenAPI Specifications and will only return a 2xx response if they are as per API Specifications. Specmatic returns 4xx response if the expectation json is not as per the OpenAPI Specifications. - -### Anatomy of a Component / API Test - -Anatomy of a Component / API Test - -Please see this [video](https://youtu.be/U5Agz-mvYIU?t=998) for reference. - -The above image shows how Specmatic Smart Mocking fits into your Component Test. A good component test isolates the system / component under test from its dependencies. Here Specmatic is emulating the dependencies of the mobile application thereby isolating it. - -**API Tests are just Component Tests where the System Under Test is a Service / API**. Here is an [example](https://github.com/specmatic/specmatic-order-bff-java/blob/main/src/test/kotlin/com/component/orders/ApiTests.kt) of how you can leverage Specmatic dynamic mocking in an API Test. Below are the pieces involved. -* **System Under Test** - [Find Available Products Service](https://github.com/specmatic/specmatic-order-bff-java/blob/main/src/main/kotlin/com/component/orders/controllers/Products.kt) - Invokes products API to get all products and filters out products where inventory is zero. -* **Dependency** - Products API mocked by Specmatic. Specmatic is set up to leverage [OpenAPI Specification of Products API](https://github.com/specmatic/specmatic-order-contracts/blob/main/io/specmatic/examples/store/openapi/api_order_v3.yaml) in the [central contract repo](https://github.com/specmatic/specmatic-order-contracts) through [specmatic.yaml](https://github.com/specmatic/specmatic-order-bff-java/blob/main/src/test/resources/specmatic.yaml) configuration. - -Let us analyse each phase of this API test. -* **Arrange** - In this step, we set up Specmatic stub server with expectation json through Specmatic http endpoint to emulate the Products API to return expected response. We also verify that Specmatic has accepted this expectation data by asserting that the response code is 2xx. This confirms that are our expectation data is in line with the OpenAPI Specification of Products OpenAPI Specification. -* **Act** - Here the test invokes System Under Test to exercise the functionality we need to test. This inturn results in the System Under Test invoking its dependency (Products Service) which is being emulated by Specmatic. Specmatic returns the response we have set up in the previous step to the System Under Test. System Under Test processes this data and responds to API Test. -* **Assert** - We now verify the response from System Under Test to ascertain if it has returned the correct response. - -## Programmatically Starting Stub Server Within Tests - -{% tabs virtualization %} -{% tab virtualization java %} -If your tests are written in a JVM based language, you can start and stop the stub server within your tests programmatically. - -Add `specmatic-core` jar dependency with scope set to test since this need not be shipped as part of your production deliverable. - -``` - - io.specmatic - specmatic-core - {{ site.specmatic-core-version }} - test - -``` - -Now you can import the utility to create the stub server. Below code snippets are in Kotlin. However, the overall concept is the same across all JVM languages such as Clojure, Scala or plain Java. - -```kotlin -import io.specmatic.stub.createStub -``` - -This utility can now be used in your test ```setup``` / ```beforeAll``` method to start the stub server. Specmatic automatically looks for your [Specmatic configuration](/references/configuration.html) file in project root directory / classpath to locate your API Specification files that need to run as part of the stub server. - -```kotlin -@BeforeAll -@JvmStatic -fun setUp() { - stub = createStub() -} -``` - -We can now programmatically set [dynamic expectations](#dynamic-expectations-aka-dynamic-stubbing-or-mocking---setting-expectations-over-specmatic-http-api) on the ```stub``` with the ```setExpectation()``` method where `````` is in the same format as [seen here](#externalizing-example-data) - -```java -stub.setExpectation(expectationJson); -``` - -If you have several such JSON expectation files that you would like to set up at once, you can pass a list of files or dir containing these expectation JSON files while creating the stub. - -```kotlin -httpStub = createStub(listOf("./src/test/resources")) -``` - -The above `createStub()` function creates your Specmatic HTTP stub with default host, port, etc. Below is an example with all values being passed in - -```kotlin -@BeforeAll -@JvmStatic -fun setUp() { - stub = createStub(listOf("./src/test/resources"), "localhost", 8090, strict = true) -} -``` - -The last parameter (`strict = true`), enables **strict** mode where Specmatic HTTP Stub will only respond to requests where expectations have been set. For any other requests, `400 Bad Request` is returned. - -And subsequently once your tests are done, you can shut down the stub server as part of your ```teardown``` / ```afterAll``` method. - -```kotlin -@AfterAll -@JvmStatic -fun tearDown() { - service?.stop() - stub.close() -} -``` - -Here is a complete example of Specmatic Contract Tests that leverages the above technique. - -[Kotlin Example](https://github.com/specmatic/specmatic-order-bff-java/blob/main/src/test/kotlin/com/component/orders/contract/ContractTests.kt) - -Please note that this is only a utility for the purpose of convenience in Java projects. Other programming languages can simply run the Specmatic standalone executable just as easily. If you do happen to write a thin wrapper and would like to contribute the same to the project, please refer to our [contribution guidelines](https://github.com/specmatic/specmatic/blob/main/CONTRIBUTING.md). - -{% endtab %} -{% tab virtualization python %} -If your tests are written in Python, you can start and stop the stub server within your tests programmatically. - -1. **Install the Specmatic Python library**: Use pip, a package installer for Python, to install the Specmatic library. - - ```bash - pip install specmatic - ``` - -2. **Run Tests with a Stub**: If you want to run the tests with a stub, you can do so like this: - - ```python - import os - from specmatic.core.specmatic import Specmatic - from your_project import app - PROJECT_ROOT_DIR = os.path.dirname(os.path.abspath(__file__)) - app_host = "127.0.0.1" - app_port = 5000 - stub_host = "127.0.0.1" - stub_port = 5000 - expectation_json_file = PROJECT_ROOT_DIR + '/test/data/expectation.json' - - Specmatic() \ - .with_project_root(PROJECT_ROOT_DIR) \ - .with_stub(stub_host, stub_port, [expectation_json_file]) \ - .with_wsgi_app(app, app_host, app_port) \ - .test(TestContract) \ - .run() - ``` - - In this example, we are passing an instance of wsgi app like flask. `stub_host`, `stub_port`, and `expectation_json_file` are the host and port for the stub server, and the path to a JSON file containing expectations for the stub, respectively. Replace `app` with your Flask application object. - - Here are complete example of [Specmatic stub server](https://github.com/specmatic/specmatic-order-bff-python/blob/main/test/test_contract.py) usage in Python. -{% endtab %} -{% endtabs %} - -## Transient Expectations (a.k.a. Transient Stubs) - -A transient mock disappears immediately after it has been exercised. - -### Setting transient expectations - -For example, create this stub file for products-api.yaml contract: - -```json -{ - "http-stub-id": "123", - "http-request": { - "method": "GET", - "path": "/storestatus", - }, - "http-response": { - "status": 200, - "body": "open" - } -} -``` - -Make an HTTP request to http://localhost:9000/storestatus with the GET method. The response says "open". - -Now try the same request again. The response is a randomized string. - -This is useful particularly when there are no distinguishing features of the request like in the above example, and we need to simulate a succession of calls to that API giving different responses. - -### Clearing Transient Expectations - -If the test fails and you need to start a new run of the test, you may need to clear all the transient mocks so that the two tests do not step on each other's toes. - -To do that, make an API call to the path /_specmatic/http-stub/ with the DELETE verb. - -To clear the transient mock in the above example, you would call http://localhost:9000/_specmatic/http-stub/123 with the DELETE verb. - ## Externalised Response Generation There may be circumstances where we need to compute the response or part of it based on the request in the expectation. Here is an example. From 66cd16bfcd8bc52ed58c1062921018c790b16f97 Mon Sep 17 00:00:00 2001 From: Joel Rosario Date: Sun, 30 Nov 2025 12:51:59 +0000 Subject: [PATCH 5/6] Updated matcher documentation --- contract_driven_development/service_virtualization.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/contract_driven_development/service_virtualization.md b/contract_driven_development/service_virtualization.md index 5059e058c..2838f109a 100644 --- a/contract_driven_development/service_virtualization.md +++ b/contract_driven_development/service_virtualization.md @@ -1592,6 +1592,7 @@ One of the most valuable applications of matchers is **resilience testing**, for #### First 2 Requests - Simulated Latency The stub introduces a 5-second delay before responding. The goal is to prove that your implementation does not wait the full 5 seconds. + It should ideally timeout earlier (e.g., at 2 seconds) and release the connection. ```json @@ -1613,13 +1614,15 @@ It should ideally timeout earlier (e.g., at 2 seconds) and release the connectio }, } } - ``` +Transient requests take priority over others. So this example will be matched first for the first 2 requests with any `amount` value. + #### Requests After Exhaustion - Recovery -Once the first matcher exhausts (after 2 requests), the next example takes over. -This simulates the downstream service recovering its performance, allowing your implementation to process requests normally. +Once the first example gets exhausted (after 2 requests), it will match no more requests. Then the next example kicks in. + +We'd also have an example available that simulates the downstream service recovering its performance, allowing your implementation to process requests normally. ```json { From b800dd299b73bebb15e59698117ce7e467dc618a Mon Sep 17 00:00:00 2001 From: Joel Rosario Date: Sun, 30 Nov 2025 13:17:28 +0000 Subject: [PATCH 6/6] Updates to matcher documentation --- .../service_virtualization.md | 103 +++++++++--------- 1 file changed, 52 insertions(+), 51 deletions(-) diff --git a/contract_driven_development/service_virtualization.md b/contract_driven_development/service_virtualization.md index 2838f109a..f37bb7355 100644 --- a/contract_driven_development/service_virtualization.md +++ b/contract_driven_development/service_virtualization.md @@ -49,13 +49,10 @@ Service Virtualization * [Request Matchers](#request-matchers) * [Matcher: `$neq`](#matcher-neq) * [Matcher: `$match`](#matcher-match) - * [Parameter: exact](#parameter-exact) - * [Parameter: datatype](#parameter-datatype) - * [Parameters: `value` and `times`](#parameters-value-and-times) - * [Match each unique value a specified number of times](#match-each-unique-value-a-specified-number-of-times) - * [Match any value a specified number of times](#match-any-value-a-specified-number-of-times) - * [Multiple matchers with **value** and **times**](#multiple-matchers-with-value-and-times) - * [Recap of `value` and `times`](#recap-of-value-and-times) + * [Match an specific value a fixed number of times](#match-an-specific-value-a-fixed-number-of-times) + * [Match any unique value a fixed number of times](#match-any-unique-value-a-fixed-number-of-times) + * [Match any value a fixed number of times](#match-any-value-a-fixed-number-of-times) + * [Multiple `value: each` matchers working together](#multiple-value-each-matchers-working-together) * [Simulating Downstream Dependency Failures using matchers](#simulating-downstream-dependency-failures-using-matchers) * [First 2 Requests - Simulated Latency](#first-2-requests---simulated-latency) * [Requests After Exhaustion - Recovery](#requests-after-exhaustion---recovery) @@ -518,7 +515,7 @@ For example, suppose that in the request we expect, the important values to be m Let's see how we can formulate an example that meets these requirements. -- Create a new file in the `employees_examples` directory named `any_name.json`: +- Create a new file in the `employees_examples` directory named `any_name.json` with the following contents: ```json { @@ -1302,7 +1299,8 @@ To clear the transient mock in the above example, you would call http://localhos Specmatic provides powerful matchers that offer dynamic, fine-grained control over how your stub server responds to incoming requests, enabling you to simulate complex and evolving scenarios. -{: .note} Requests are validated against the specification first, before any matchers in examples are evaluated. +{: .note} +Requests are validated against the specification first, before any matchers in examples are evaluated. ### Matcher: `$neq` @@ -1334,7 +1332,7 @@ For example, the following example matches only requests where `status` is not e This matcher provides flexible options to control how request fields match example data and how many times a particular example can be used. -#### Match an exact value a fixed number of times +#### Match an specific value a fixed number of times Consider the following **transient** example: @@ -1361,7 +1359,7 @@ This matcher will respond to two requests where `message` is exactly `hello`. Af **Example:** -Request 1: +Request 1 payload: ```json { @@ -1369,7 +1367,7 @@ Request 1: } ``` -Request 2: +Request 2 payload: ```json { @@ -1379,7 +1377,7 @@ Request 2: After these two requests, the matcher becomes **exhausted**, and this transient example will not match further requests such as: -Request 3: +Request 3 payload: ```json { @@ -1389,7 +1387,7 @@ Request 3: If there are multiple matchers in an example, the example remains active until **all** of its matchers are exhausted. -##### Each unique value matched a fixed number of times +#### Match any unique value a fixed number of times Consider the following **transient** example: @@ -1414,19 +1412,19 @@ Consider the following **transient** example: Each unique value for `text` can be matched twice. For example: -**Request payload 1:** `{ "text": "hello" }` +Request 1 payload: `{ "text": "hello" }` -**Request payload 2:** `{ "text": "hello" }` +Request 2 payload: `{ "text": "hello" }` After these two requests, the matcher for `text` with value `hello` becomes **exhausted**. -**Request payload 3:** `{ "text": "world" }` +Request 3 payload: `{ "text": "world" }` The value "world" has never been seen before, and hence this matcher will be active for "world", and will match it twice before getting exhausted. If there are multiple matchers in an example, the example remains active until **all** of its matchers are exhausted. -##### Any value matched a fixed number of times +#### Match any value a fixed number of times Consider this **transient** example: @@ -1453,9 +1451,9 @@ Here, the matcher can match any value twice, regardless of what it is. After two #### Multiple `value: each` matchers working together -In a previous section we looked at an example with a single `value: each` matcher. But now, let's see what happens when there are multiple `value: each` matchers work. +Specmatic actually tracks the usage of each unique combination of matcher values in the presence of `value: each`. This can be better understood with an example which has multiple matching having `value: each`. -Consider this **transient** example where both `id` and `details` use `value: each` with `times: 1`: +Consider this **transient** example where both `id` and `details` use `value: each` with `times: 1`. ```json { @@ -1477,17 +1475,14 @@ Consider this **transient** example where both `id` and `details` use `value: ea } ``` -When there are multiple `value: each` matchers in an example, Specmatic tracks the usage of each unique combination of matcher values. - Let's walk through a sequence of requests. -**Initial state:** +Initial state: -* No `(id, details)` pairs have been used. -* `id` matcher: all values are **Active**. -* `details` matcher: all values are **Active**. +* `id` matcher: **Active**. +* `details` matcher: **Active**. -**Request 1** +Request 1 payload: ```json { @@ -1498,11 +1493,12 @@ Let's walk through a sequence of requests. This is the first time the stub sees `id: 10` together with `details: "packed"`, so it matches. -**Matcher state after Request 1:** +Matcher state after Request 1: -* Pair `id: 10`, `details: "packed"`: used 1/1 (**Exhausted** for this pair) +* `id` matcher with value `10` given `details: "packed"`: used `/` (**Exhausted**) +* `details` matcher with value `packed` given `id: 0`: used `/` (**Exhausted**) -**Request 2** +Request 2 payload ```json { @@ -1513,12 +1509,11 @@ This is the first time the stub sees `id: 10` together with `details: "packed"`, This is the exact same pair as Request 1. -**Matcher state after Request 2:** +Matcher state after Request 2: * Pair `id: 10`, `details: "packed"` is already 1/1 (**Exhausted**) → this request will **not** match this example. -**Request 3** - +Request 3 payload ```json { "id": 10, @@ -1528,14 +1523,16 @@ This is the exact same pair as Request 1. This is a new pair: `id: 10` with `details: "shipped"`. -**Matcher state after Request 3:** +Matcher state after Request 3: -* Pair `id: 10`, `details: "packed"`: **Exhausted**. -* Pair `id: 10`, `details: "shipped"`: **Exhausted**. +* `id` matcher with value `10` given `details: "packed"`: used `/` (**Exhausted**) +* `id` matcher with value `10` given `details: "shipped"`: used `/` (**Exhausted**) +* `details` matcher with value `packed` given `id: 10`: used `/` (**Exhausted**) +* `details` matcher with value `shipped` given `id: 10`: used `/` (**Exhausted**) -Even though `id = 10` was seen before, this request works because this exact combination (`10` + `"shipped"`) had not been used previously. +Even though `id = 10` was seen before, this request worked because this exact combination (`10` + `"shipped"`) had not been used previously. But with `times: 1`, this combination is now exhausted. -**Request 4** +Request 4 payload: ```json { @@ -1544,14 +1541,16 @@ Even though `id = 10` was seen before, this request works because this exact com } ``` -This reuses the pair from Request 3. - -**Matcher state after Request 4:** +This reuses the pair from Request 3. But given that these values have already been used together once, this request will not match. -* Pair `id: 10`, `details: "shipped"` is already 1/1 (**Exhausted**) → this request does **not** match this example. +Matcher state after Request 4: -**Request 5** +* `id` matcher with value `10` given `details: "packed"`: used `/` (**Exhausted**) +* `id` matcher with value `10` given `details: "shipped"`: used `/` (**Exhausted**) +* `details` matcher with value `packed` given `id: 10`: used `/` (**Exhausted**) +* `details` matcher with value `shipped` given `id: 10`: used `/` (**Exhausted**) +Request 5 payload ```json { "id": 20, @@ -1561,11 +1560,14 @@ This reuses the pair from Request 3. Now we have a new pair: `id: 20` with `details: "packed"`. -**Matcher state after Request 5:** +Matcher state after Request 5: -* Pair `id: 10`, `details: "packed"`: **Exhausted**. -* Pair `id: 10`, `details: "shipped"`: **Exhausted**. -* Pair `id: 20`, `details: "packed"`: used 1/1 (**Exhausted** for this pair). +* `id` matcher with value `10` given `details: "packed"`: used `/` (**Exhausted**) +* `id` matcher with value `10` given `details: "shipped"`: used `/` (**Exhausted**) +* `id` matcher with value `20` given `details: "packed"`: used `/` (**Exhausted**) +* `details` matcher with value `packed` given `id: 10`: used `/` (**Exhausted**) +* `details` matcher with value `shipped` given `id: 10`: used `/` (**Exhausted**) +* `details` matcher with value `packed` given `id: 20`: used `/` (**Exhausted**) This request matches because, although `"packed"` was seen earlier with `id = 10`, the combination `id: 20`, `details: "packed"` is new. @@ -1575,11 +1577,10 @@ In summary: * Reusing the same pair again will not match the example. * Reusing either `id` or `details` with a new partner is allowed until that new combination has also been used once. -**Key takeaway:** +### Note On Matcher Exhaustion -* Each matcher (`name` and `type`) tracks its own count independently. -* Once a matcher reaches its `times` limit, it moves from **Active** to **Exhausted** and stops matching. -* When all matchers in a transient example are exhausted, the example itself is considered exhausted and will no longer be used to match incoming request payloads. +* An example will continue to match incoming requests as long as at least one of its matchers is not **exhausted** for the given request values. +* When all matchers in a transient example are exhausted for the incoming request, the example itself is considered **exhausted** and will not return match success, even though the actual values in the request align with the data in the example. ### Simulating Downstream Dependency Failures using matchers