LLM Hallucination Detection Service

Using this project you can deploy your own REST API endpoint for detecting hallucination in LLM generated text.

Completely Open Source and Free to use.

Usage

This project exposes a REST API endpoint that listens for queries to detect LLM hallucination in generated text. The API endpoint accepts a JSON payload with the following schema:

{
  "data": {
    "query": "Prompt: <prompt text> \n Generated response: <LLM response text> \n Instructions for response generation: <instructions>"
  }
}

The API endpoint then performs hallucination detection checks on the LLM generated response and returns the result in the following schema:

{
  "factualConsistency": <true | false>,
  "factualConsistencyReasoning": "Reasoning behind the result of the factual consistency check",
  "instructionAdherence": <true | false>,
  "instructionAdherenceReasoning": "Reasoning behind the result of the instruction adherence check",
  "contextRelevance": <true | false>,
  "contextRelevanceReasoning": "Reasoning behind the result of the context relevance check",
  "error": null // Optional error message if an error occurs during processing
}

Following checks are performed:

1. Factual Consistency Detection: checks that the response generated by LLM is factually correct.
2. Instruction Adherence Check: checks that the response generated by LLM adheres to the instructions provided.
3. Context Relevance Check: checks that the response generated by LLM is relevant to the context of the prompt.

Underlying Architecture

This project uses the open-source QvikChat framework to build the service endpoint.

The core element of this project is the system prompt that it uses to generate the hallucination detection response.

You can find the code for the endpoint and server configurations in the src/index.ts file, and the system prompt in the src/prompts.ts file.

You can add API-key based authentication, response caching, and other features to the service endpoint through QvikChat. Check out the QvikChat Documentation for more information.

Getting Started

Simply, clone the TypeScript Project Starter Template to get started.

git clone https://github.com/pranav-kural/llm-hallucination-detection-service.git

Install Dependencies

Install the dependencies using PNPM:

pnpm install

Compile Code

Use SWC to compile your TypeScript code:

pnpm build

Pre-configured to output the compiled code to the lib directory.

Run the Code

Run the compiled code:

pnpm start

Pre-configured to run the index.js file in the lib directory.

Development

You can also use the dev script instead when making changes to the source code. This script will compile your code and run it in watch mode:

pnpm dev

Pre-configured to watch the src directory for changes and recompile the code automatically. Please that this uses tsx and not swc for compilation.

Testing

Once you have the server running, you can use the below curl commands to test the hallucination detection service:

curl -X POST "http://127.0.0.1:3400/detect" -H "Content-Type: application/json"  -d '{"data": { "query": "Prompt: What is the capital of India? \n Generated response: The capital of India is Mumbai \n Instructions for response generation: Response must start with the phrase \"Sure, here is..\"" } }'

For the above query, some of the checks will fail.

You can test with the following query to see all checks pass:

curl -X POST "http://127.0.0.1:3400/detect" -H "Content-Type: application/json"  -d '{"data": { "query": "Prompt: Explain the theory of general relativity in simple terms. Generated response: The theory of general relativity, proposed by Albert Einstein, explains that gravity is not just a force but a curvature of space and time caused by massive objects. The more massive the object, the more it warps the space around it, which affects how objects move. Instructions for response generation: The explanation should be simple and accurate, avoiding complex scientific jargon." } }'

Genkit Developer UI

You could also start the Genkit Developer UI instead of running your own server for testing, using the following command:

npx genkit start

For more information on how to install the Firebase Genkit CLI, please check the Genkit CLI documentation.

Once you have the Genkit Developer UI loaded in your browser, you should be able to see the detect endpoints under the Flows section in the left sidebar. Simply click on the endpoint, and enter the query you want to test with. Clicking the Run button will send the query to the endpoint and the response generation process will start.

Linting

Pre-configured with ESLint to enforce code quality and consistency. You can run ESLint using the following command:

pnpm lint

Formatting

Pre-configured with Prettier to format your code automatically. You can run Prettier using the following command:

pnpm format

Pre-deploy Workflow

An npm script predeploy is defined the package.json file to run the following in sequence:

1. lint: Run ESLint to enforce code quality and consistency.
2. format: Run Prettier to format your code automatically.
3. build: Compile your TypeScript code using SWC.
4. test: Run Jest to ensure your code works as expected.

pnpm predeploy

GitHub Actions

Pre-configured with GitHub Actions to run your tests and lint your code automatically on every push. You can find the workflow configuration in the .github/workflows directory. Also, includes setup for dependabot to keep your dependencies up-to-date. Please note that, you may need to enable dependabot in your repository settings. Learn more about Dependabot.

Git Setup

After cloning the repository, you can update the remote URL to point to your repository, or remove the existing git commit history and start fresh.

Update the remote URL to point to your repository:

git remote set-url origin

Remove the existing git commit history and start fresh:

rm -rf .git
git init
git add .
git commit -m "Initial commit"
git branch -M main
git remote add origin <your-repository-url>
git push -u origin main

TypeScript Project Starter Template

This project was scaffolded using the TypeScript Project Starter Template. It comes pre-configured with the following features:

• TypeScript: TypeScript to allow you to write type-safe code efficiently.
• ESLint: ESLint to enforce code quality and consistency.
• Prettier: Prettier to format your code automatically and ensure consistent code style.
• Jest: Jest to run your tests and ensure your code works as expected.
• SWC: For faster and more efficient TypeScript compilation.
• PNPM: PNPM to manage your dependencies efficiently.
• GitHub Actions: GitHub Actions to run your tests and lint your code automatically on every push.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Issues

If you encounter any issues or bugs while using this project, please report them by following these steps:

1. Check if the issue has already been reported by searching our issue tracker.
2. If the issue hasn't been reported, create a new issue and provide a detailed description of the problem.
3. Include steps to reproduce the issue and any relevant error messages or screenshots.

Open Issue