Post cover image

February 22, 2026

Building a Cost-Free Local Hybrid RAG Knowledge Base with LangChain4j and OpenSearch

Build hybrid RAG with LangChain4j + OpenSearch: BM25 + k-NN fusion for GitHub docs. Java, Docker, REST-verified. Step-by-step guide.

By Dennis Lee

7 min read

Hybrid RAG delivers better results than vector-only. Build a GitHub-powered KB with LangChain4j + OpenSearch: BM25 + k-NN fusion, Java, Docker, REST-verified. Step-by-step guideline.

Why Hybrid RAG Beats Standard RAG

Before we dive into the code, let's address the elephant in the room: why hybrid retrieval?

Standard RAG (Retrieval-Augmented Generation) relies solely on vector similarity search. While powerful for semantic matching, it has blind spots:

Standard Vector-Only RAG

❌ Misses exact keyword matches (e.g., error codes, API names)

❌ Struggles with rare terms or technical jargon

❌ Single retrieval strategy = single point of failure

❌ Lower recall for multi-faceted queries

Hybrid RAG (Text + k-NN)

✅ Combines BM25 keyword scoring with vector similarity

✅ Preserves lexical precision for technical documentation

✅ Reciprocal Rank Fusion (RRF) balances both signals

✅ Higher relevance across diverse query types

This guide walks you through building exactly that: a GitHub documentation knowledge base using LangChain4j and OpenSearch.

Prerequisites

  • JDK 21 (required by the project) — verify with java --version
  • Apache Maven 3.8+ — verify with mvn -v
  • Docker & Docker Compose — for OpenSearch deployment
  • GitHub Personal Access Token with repo read scope
  • Environment variable: export GITHUB_TOKEN=your_token_here

Step 1: Deploy OpenSearch with Docker Compose

The repo provides a production-ready docker-compose.yml that configures OpenSearch with the k-NN plugin enabled and security disabled for local development:

# docker-compose.yml
services:
    opensearch-node1:
        image: opensearchproject/opensearch:latest
        container_name: opensearch-node1
        environment:
            - cluster.name=opensearch-cluster
            - node.name=opensearch-node1
            - discovery.type=single-node
            - bootstrap.memory_lock=true
            - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
            - OPENSEARCH_INITIAL_ADMIN_PASSWORD=8vN@zW5&yT1p
            - DISABLE_SECURITY_PLUGIN=true
            - DISABLE_INSTALL_DEMO_CONFIG=true
        ulimits:
            memlock:
                soft: -1
                hard: -1
            nofile:
                soft: 65536
                hard: 65536
        volumes:
            - opensearch-data1:/usr/share/opensearch/data
        ports:
            - 9200:9200  # REST API
            - 9600:9600  # Performance Analyzer
        networks:
            - opensearch-net
    opensearch-dashboards:
        image: opensearchproject/opensearch-dashboards:latest
        container_name: opensearch-dashboards
        ports:
            - 5601:5601
        environment:
            - 'OPENSEARCH_HOSTS=["http://opensearch-node1:9200"]'
            - DISABLE_SECURITY_DASHBOARDS_PLUGIN=true
        networks:
            - opensearch-net

volumes:
    opensearch-data1:

networks:
    opensearch-net:

Launch the stack:

cd 01_RAG
docker compose up -d
None

✅ Verification: Confirm OpenSearch is healthy:

curl -X GET "http://localhost:9200/_cluster/health?pretty"

Expected response:

None

Step 2: Configure Maven Dependencies

The pom.xml declares all required LangChain4j modules and OpenSearch clients:

<dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>dev.langchain4j</groupId>
                <artifactId>langchain4j-bom</artifactId>
                <version>1.11.0</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>

            <dependency>
                <groupId>com.fasterxml.jackson</groupId>
                <artifactId>jackson-bom</artifactId>
                <version>2.17.2</version> <!-- or your preferred stable version -->
                <type>pom</type>
                <scope>import</scope>
            </dependency>

            <dependency>
                <groupId>org.apache.httpcomponents.client5</groupId>
                <artifactId>httpclient5-parent</artifactId>
                <version>5.4.3</version> <!-- Managed version for all HC5 modules -->
                <type>pom</type>
                <scope>import</scope>
            </dependency>

        </dependencies>
    </dependencyManagement>
<!-- 01_RAG/pom.xml (key dependencies) -->
<dependencies>
    <!-- Core LangChain4j -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j</artifactId>
    </dependency>

    <!-- GitHub Document Loader -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-document-loader-github</artifactId>
    </dependency>

    <!-- OpenSearch Integration -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-opensearch</artifactId>
    </dependency>

    <!-- Embedding Model (local, no API key needed) -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-embeddings-all-minilm-l6-v2</artifactId>
    </dependency>

    <!-- Low-level OpenSearch Java Client for index management -->
    <dependency>
        <groupId>org.opensearch.client</groupId>
        <artifactId>opensearch-java</artifactId>
        <version>3.6.0</version>
    </dependency>
</dependencies>

Step 3: Create a Hybrid-Ready OpenSearch Index

Hybrid search requires an index that supports both full-text (BM25) and k-NN vector search. Create it via REST API:

# Create index with hybrid mapping
curl -X PUT "http://localhost:9200/sample-index" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "index": {
      "knn": true,
      "knn.algo_param.ef_search": 100
    }
  },
  "mappings": {
    "properties": {
      "text": {
        "type": "text",
        "analyzer": "standard"
      },
      "vector": {
        "type": "knn_vector",
        "dimension": 384,
        "method": {
          "name": "hnsw",
          "space_type": "cosinesimil",
          "engine": "nmslib"
        }
      },
      "metadata": {
        "type": "object"
      }
    }
  }
}'

http://localhost:5601/app/opensearch_index_management_dashboards#/indices?from=0&search=&showDataStreams=false&size=20&sortDirection=desc&sortField=index

None

✅ Verify index mapping:

curl -X GET "http://localhost:9200/sample-index/_mapping?pretty"
None

✅ Confirm k-NN plugin is active:

curl -X GET "http://localhost:9200/_cat/plugins" | grep knn
None

Step 4: Load GitHub Documentation & Ingest to OpenSearch

The application uses LangChain4j's GitHubDocumentLoader to fetch markdown files from a target repository. Here's the core ingestion flow (based on Application.java structure from the repo):

public class Application {
    public static void main(String[] args) {
        // 1. Initialize embedding model (local All-MiniLM-L6-v2)
        EmbeddingModel embeddingModel = new AllMiniLmL6V2EmbeddingModel();

        // 2. Configure OpenSearch connection
        OpenSearchConnection osConn = new OpenSearchConnection("http://localhost:9200");

        // 3. Load documents from GitHub
        String githubToken = System.getenv("GITHUB_TOKEN");
        List<Document> documents = gitHubDocumentLoader.loadDocuments(
            "${github_repo_owner}",
            "${github_repo}",
            "main",
            "${github_repo_path}",
            new TextDocumentParser()
        );

        // 4. Generate embeddings and ingest
        String index = "sample-index";

        EmbeddingModel embeddingModel = new AllMiniLmL6V2EmbeddingModel();

        EmbeddingStore<TextSegment> embeddingStore = OpenSearchEmbeddingStore.builder()
            .serverUrl("http://localhost:9200")
            .indexName(index)
            .build();

        EmbeddingStoreIngestor embeddingStoreIngestor = EmbeddingStoreIngestor.builder()
            .embeddingModel(embeddingModel)
            .embeddingStore(embeddingStore)
            .build();

        embeddingStoreIngestor.ingest(documents);
    }
}

✅ Verify ingestion via OpenSearch REST API:

# Check document count
curl -X GET "http://localhost:9200/sample-index/_count?pretty"
None

This can also be observed at the dashboard: http://localhost:5601/app/opensearch-query-workbench#/

None 

# Sample search to confirm text field works
curl -X GET "http://localhost:9200/sample-index/_search?pretty" -H 'Content-Type: application/json' -d'
{
  "query": {
    "match": {
      "text": "embedding model"
    }
  },
  "size": 2
}'
None

Step 5: Execute Hybrid Search with Reciprocal Rank Fusion

The repo's HybridContentRetriever.java implements the critical fusion logic. Hybrid retrieval combines:

  1. BM25 keyword search on the text field
  2. k-NN vector search on the embedding field
  3. Reciprocal Rank Fusion (RRF) to merge results

First create the RRF pipeline in OpenSearch:

curl --location --request PUT 'http://localhost:9200/_search/pipeline/rrf-pipeline' \
--header 'Content-Type: application/json' \
--data '{
  "description": "Post processor for hybrid RRF search",
  "phase_results_processors": [
    {
      "score-ranker-processor": {
        "combination": {
          "technique": "rrf"
        }
      }
    }
  ]
}'
None

Now to invoke the query:

public record HybridContentRetriever(OpenSearchClient client,
                                     EmbeddingModel embeddingModel,
                                     String indexName) implements ContentRetriever {

    private final static Logger log = LoggerFactory.getLogger(HybridContentRetriever.class);

    // ...

    @Override
    public List<Content> retrieve(final Query query) {
        try {
            List<org.opensearch.client.opensearch._types.query_dsl.Query> queries = new ArrayList<>();

            MatchQuery matchQuery = new MatchQuery.Builder()
                .field("text")
                .query(FieldValue.of(query.text()))
                .build();
            queries.add(org.opensearch.client.opensearch._types.query_dsl.Query.of(q -> q.match(matchQuery)));

            SearchResponse<ObjectNode> searchResponse = client.search(s -> s
                    .index(this.indexName)
                    .searchPipeline("rrf-pipeline")
                    .query(q -> q.hybrid(h -> h
                        .queries(
                            textQuery -> textQuery.match(m -> m
                                .field("text")
                                .query(FieldValue.of(query.text()))
                        )
                        .queries(
                            knnQuery -> knnQuery.knn(kn -> kn
                                .field("vector")
                                .vector(this.embeddingModel.embed(query.text()).content().vectorAsList())
                                .k(10)
                        )
                    ))
                , ObjectNode.class);

            return searchResponse.hits()
                .hits()
                .stream()
                .map(it -> {
                    ObjectNode source = it.source();
                    String text = source.has("text")
                        ? source.get("text").asText()
                        : "";

                    return Content.from(text);
                })
                .toList();
        } catch (IOException e) {
            throw new RuntimeException("Failed to retrieve content from OpenSearch", e);
        }
    }
}

✅ Test hybrid search via REST API (OpenSearch native hybrid query):

HttpClient.Builder clientBuilder = HttpClient.newBuilder()
    .version(HttpClient.Version.HTTP_1_1); // Force HTTP/1.1

JdkHttpClientBuilder jdkHttpClientBuilder = JdkHttpClient.builder()
    .httpClientBuilder(clientBuilder);

ChatModel chatModel = OpenAiChatModel.builder()
    .apiKey("lm-studio")
    .baseUrl("http://192.168.1.17:1234/v1")
    .apiKey("lm-studio")
    .modelName("phi-3-mini-4k-instruct")
    .httpClientBuilder(jdkHttpClientBuilder)
    .build();

HybridContentRetriever hybridContentRetriever = new HybridContentRetriever(
    openSearchClient,
    embeddingModel,
    index
);

Assistant assistant = AiServices.builder(Assistant.class)
    .chatModel(chatModel)
    .chatMemory(MessageWindowChatMemory.withMaxMessages(10))
    .contentRetriever(hybridContentRetriever)
    .build();

String answer = assistant.chat("Recommend a DDD java folder structure?");

System.out.println(answer);

Step 6: Run the Full Demo

Execute the application with your GitHub token.

Here's an example using the GitHub repo https://github.com/dennisholee/ArchitectureDojo to answer the above query.

Here’s a **Domain-Driven Design (DDD) compliant Java folder structure** based on best practices, adapted from your context and common DDD patterns. This structure emphasizes separation of concerns, modularity, and alignment with domain-driven principles:

---

### **Root Directory**

src/ ├── main/ │ ├── java/ # Core application packages │ │ ├── com/example/ddd/ # Example namespace (adjust as needed) │ │ │ ├── config/ # Configuration-related classes │ │ │ ├── adapter/ # External adapters (APIs, databases, etc.) │ │ │ │ ├── api/ # REST/gRPC clients (e.g., io.forest.ddd.adapter.api.client) │ │ │ │ │ ├── client/ # API clients (DTOs, interceptors) │ │ │ │ │ └── dto/ # Data Transfer Objects │ │ │ │ ├── database/ # Database adapters (JPA/Hibernate, etc.) │ │ │ │ ├── kafka/ # Kafka producers/consumers (e.g., DomainEventAVRO) │ │ │ │ └── event/ # Event-driven components │ │ │ │ └── domain-event/ │ │ │ ├── application/ # Application layer (use cases, commands) │ │ │ │ ├── command/ # DTOs for commands (e.g., AddClaimCommand) │ │ │ │ ├── query/ # DTOs for queries (e.g., GetClaimsByIdQuery) │ │ │ │ └── usecase/ # Application services │ │ │ ├── domain/ # Core domain logic (entities, value objects) │ │ │ │ ├── entity/ # Domain entities (e.g., Claim, Customer) │ │ │ │ ├── repository/ # Repository interfaces (e.g., IClaimRepository) │ │ │ │ └── value-object/ │ │ │ ├── infrastructure/ # Infrastructure-specific implementations │ │ │ │ ├── persistence/ # JPA/Hibernate mappings, etc. │ │ │ │ └── kafka/ # Kafka producer/consumer implementations │ │ │ └── exception/ # Custom exceptions (e.g., DomainException) │ │ │ └── business/ │ │ ├── resources/ # Static resources (Swagger/OpenAPI configs) │ │ │ ├── swagger/ # OpenAPI/Swagger definitions │ │ │ │ ├── client/ # API client specs (e.g., api.yml) │ │ │ │ └── server/ # Gateway/server specs (e.g., medicalClaims.yaml) │ │ │ └── application.properties │ │ └── test/ # Test packages │ │ ├── unit/ │ │ └── integration/ │ └── resources/ │ ├── static/ # Static assets (if any) │ └── templates/ # Thymeleaf/JSP templates (if used)

The application will:

  1. Load markdown files from the configured GitHub repo
  2. Generate embeddings using the local all-minilm-l6-v2 model
  3. Ingest documents into the hybrid OpenSearch index
  4. Execute sample hybrid queries and display the retrieved context
  5. (Optionally) Pass context to a chat model for RAG generation

Why This Architecture Wins for Technical Documentation

  1. Precision for Technical Terms: When a developer searches for @EnableRag or 409 Conflict, BM25 ensures exact matches rank highly—something pure vector search often misses.
  2. Semantic Flexibility: Queries like "how do I fix authentication errors?" retrieve relevant docs even without keyword overlap, thanks to vector similarity.
  3. Resilience to Query Variation: Hybrid retrieval handles typos, synonyms, and paraphrasing better than either approach alone.
  4. Transparent Tuning: The ALPHA parameter lets you dynamically adjust the text/vector balance per use case—critical for evolving documentation sets.
  5. Open Source & Self-Hosted: No vendor lock-in. Run everything locally with Docker, ideal for air-gapped enterprise environments.

As highlighted in recent RAG architecture discussions, combining graph, keyword, and vector retrieval in OpenSearch provides "a practical middle ground" that grounds LLM responses in verifiable source material