Lab 6: Kafka¶

Your colony is playing, but so is everyone else. Until now, your logic-service learned about the map in one way only: every turn, the game backend calls your Mission API with what your own squad sees. This lab adds a second source of information.

Part of what happens on the map leaks as radio chatter: a squad member extracting resources, an orbital scan that finds a large deposit, an attack. Rival colonies leak, and so does yours. The game backend does not send this chatter to your service with an HTTP call. It publishes it to Apache Kafka, a message broker that sits between the backend and every service that wants to listen. Your logic-service subscribes to it and reads the chatter at its own pace.

What you overhear is raw material. Your logic decides which messages teach it something new, and stores that knowledge as Points of Interest (POIs) through the DAO you built in Lab 3. Finally, receiving the chatter has to work everywhere your service runs: on your own machine, in staging and in production.

The page follows that order. It first explains Kafka and the chatter, so you can plan the work as a team. Then you set up a local broker, implement and test the consumer, and finally connect it to staging and production.

Group A

This lab is for Group A. Group B continues with Lab 7.

Goals and scope¶

Consume the Mission Chatter event stream from Apache Kafka in your logic-service.
Use it to discover Points of Interest (POI's) in your active missions without overloading your POI storage.
Test the consumer without a running broker, so your GitLab CI pipeline works too.
Connect the consumer locally (Docker Compose), in staging, and in production.

Lab 6 takes place during the competition (see Lab 5): improvements reach production through reviewed merge requests and version tags, like any release.

What is Apache Kafka?¶

Apache Kafka is a distributed event log. Producers append events to a named topic; consumers read them at their own pace. Kafka stores each event for a retention period and tracks, per consumer group, how far that group has read (its offset). Read the Kafka introduction for the vocabulary.

Concept	Meaning
Topic	A named stream of events, such as `mission-chatter`.
Partition	A topic is split into ordered partitions; ordering holds within one partition.
Offset	The position of an event in a partition.
Consumer group	Consumers sharing a `group.id` split the work and share their stored offsets.

Imagine the game backend sent chatter to your service the way it sends mission requests: one HTTP call per event, to the logic-service of every team. That would cause real problems:

Your service is being redeployed for a minute. Every call in that minute fails. Either the events are lost, or the backend has to retry and keep them for you.
Your service is slow. The backend waits for it, or builds up a queue of its own.
A new team, or a new service such as a dashboard, wants the chatter too. The backend must be changed to call it.

Kafka removes these problems by putting a broker in between. The backend publishes each event once to the broker and moves on. Your service reads from the broker whenever it is ready, like collecting mail from a mailbox instead of having someone wait at your door. The backend does not know who listens, or whether they are listening right now:

Decoupling. Your logic-service can be down, for example during a rolling deployment, and the backend does not notice. The events are buffered and delivered when you come back.
Scalability. Adding another consumer group gives another service the same events.
Reliability. Events are persisted by the broker rather than lost with a crashed process.

New logic feature: receiving Mission Chatter¶

Every interaction of a colony with the game (a move, a resource extraction, an attack, ...) has a 40% base chance of being leaked. In the game world this is a radio broadcast that anyone listening can pick up. Upgrades in both the economic and military tech trees lower the chance that a colony's actions leak.

Each leaked interaction becomes a MissionEvent on the mission-chatter topic, as JSON:

Field	Meaning
`missionId`	The mission the event occurred in.
`turnNumber`	The turn it occurred in.
`subjectEntityId`	The entity that acted.
`subjectLocation`	The tile `(x, y)` of the acting entity.
`action`	What it did. Polymorphic: the `type` field selects the concrete action.
`outcome`	What happened. Polymorphic as well, selected by `type`.
`colonyId`	The acting colony. Absent when the actor is not tied to a colony.

Update model-lib to 1.3.0

Update your model-lib dependency to 1.3.0 or newer, so your service can read the new MissionEvent.

See the model-lib Javadoc for all action and outcome types.

Examples¶

A resource extraction:

{
  "missionId": "29921f07d94cd133",
  "turnNumber": 97,
  "subjectEntityId": "7cc001986b158452",
  "subjectLocation": { "x": 96, "y": 34 },
  "action": { "type": "ExtractResourceAction" },
  "outcome": {
    "type": "ExtractResourceOutcome",
    "resourceType": "WATER",
    "depositId": "d0c1837749f596cf",
    "quantityExtracted": 4,
    "crateAfter": { "WATER": 215 },
    "depositQuantityLeft": 12,
    "largeDeposit": false
  },
  "colonyId": "colony-2"
}

A rival extracted water at (96, 34). If your colony does not know that deposit yet, it is a new POI, and the outcome says how much it holds and whether it is a large deposit. What would depositQuantityLeft: 0 tell you about it?

A large deposit that was found:

{
  "missionId": "29921f07d94cd133",
  "turnNumber": 91,
  "subjectEntityId": "718db499c00c2798",
  "subjectLocation": { "x": 31, "y": 14 },
  "action": { "type": "NoAction" },
  "outcome": {
    "type": "LargeDepositFoundOutcome",
    "colonyId": "colony-2",
    "deposit": {
      "type": "ResourceDeposit",
      "id": "718db499c00c2798",
      "location": { "x": 31, "y": 14 },
      "colonyId": null,
      "resourceType": "WATER",
      "quantity": 1236,
      "largeDeposit": true,
      "maxQuantity": 1236
    }
  },
  "colonyId": "colony-2"
}

The orbital scan of a rival found this deposit. Its full details are included, even though it may be nowhere near your squad.

An attack:

{
  "missionId": "29921f07d94cd133",
  "turnNumber": 234,
  "subjectEntityId": "9bca60c8a3ea3887",
  "subjectLocation": { "x": 97, "y": 12 },
  "action": {
    "type": "AttackAction",
    "targetLocation": { "x": 97, "y": 13 },
    "targetEntityId": "63c605b5a97a9974"
  },
  "outcome": {
    "type": "AttackOutcome",
    "damageApplied": 55,
    "targetHpLeft": 10,
    "targetColonyId": "colony-4"
  }
}

There is no colonyId: the attacker is a hostile entity, one that is probably still at (97, 12). An attacker that does have a colonyId could be a squad member or a turret, and the event does not say which.

Design considerations¶

Do not forward everything to your POI storage. Think about:

Which missions? Missions are played in sequence in a shared world, and every colony is active in the current one. The topic also carries events of missions your service never joined or that have ended, for example in a backlog after a restart. Only act on events for a mission your service currently has active. Where does your service know that from?
Which events? Decide which events your logic has a use for. The POI storage from Lab 3 was built for static points of interest, deposits and hostile organisms, so that is the baseline: which events can teach you about those? Many carry nothing new, such as your own colony's actions, which your squad already reported through the Mission API. Want to do more, for example tracking rivals that move around? You are free to, with your own model and storage, but the same care for the load it causes applies.
How many writes? Events arrive in bursts. Ten squad members extracting from one deposit are ten events, but only one fact: the deposit is there. Do not write ten times. Compact the events: of all events about the same POI, only the latest one counts. Which fields tell you two events are about the same POI? And what if the latest event says the deposit is now empty? This is the batching option from Lab 3, applied to a stream.
How do they combine with the Mission API? A deposit seen by your squad and the same deposit overheard in chatter are one POI. Reuse Lab 3's identity rule.
Which thread? The Kafka listener runs on its own threads, next to the threads handling the game backend's HTTP requests. Several of them may touch your POIs at once. What does your Lab 3 DAO guarantee?

Within a mission, you may assume events arrive in order: of two events about the same POI, the one that comes later in the stream is the latest. The backend uses the missionId as the message key, so all events of one mission end up in the same partition.

Optional: draw conclusions from history

Remember what you overheard and combine it: an entity of a rival colony that only attacks, and always from the same tile, might be a stationary turret. Only attempt this once the requirements below are met and tested.

Before splitting the work¶

Start together and settle the shared decisions:

What do you want from the chatter? Look back at the examples and the design considerations, then agree on which events are worth acting on and which you will ignore.
How do the consumer and the logic work together? The consumer receives events; your logic decides what they mean. Agree on the method (and package) the consumer calls to hand over a batch of events (see step 3), so teammates can work on both sides in parallel.
Which settings must be configurable? Topic, consumer group and broker will differ per environment. Agree on property names before anyone writes configuration.

Create work items under a Checkpoint 3 milestone for this lab, as before. A possible split: consumer and tests, filtering and POI logic, Compose, staging and production configuration. Child work items can branch off these broader categories. The deployment work is blocked on a working consumer; the logic can start against hand-written events.

Three-hour session milestone: your service receives real chatter from the local Compose Kafka in batches of MissionEvent objects, filters and compacts them into POIs stored through your DAO, and has unit tests for the filtering and compaction. The integration test, staging and production are follow-up work.

Local development setup¶

Your consumer needs a broker to talk to from the very first implementation step, so set it up before you write any code. The default configuration is enough for your service. The Compose setup from Lab 1 needs three changes.

1. Upgrade the mission-runner image. Chatter is a newer feature than the mission-runner you started in Lab 1: it is part of release 0.6.0. Your Compose file uses the latest tag, but Docker does not check the registry for a newer latest by itself. Pull it and recreate the container:

docker compose pull mission-runner
docker compose up -d mission-runner

Check that the version in the startup log is 0.6.0. If you pinned a release tag in Lab 1, change it to 0.6.0 in your docker-compose.yml instead.

2. Make the broker reachable from your machine. Kafka clients connect to the bootstrap address, then reconnect to the advertised listener the broker reports. Your Compose broker advertises only kafka:9092, a name that exists on the Compose network but not on your host. Your service, running on the host, would connect and then fail. The broker needs a second listener for clients on the host:

keep PLAINTEXT (kafka:9092) for containers, and mark it as the inter-broker listener;
add an EXTERNAL listener advertised as localhost:9092, with a security-protocol-map entry;
publish a different container port, since both listeners cannot share one.

The exact changes

Change these lines of the kafka service in docker-compose.yml:

-      KAFKA_LISTENERS: PLAINTEXT://:9092,CONTROLLER://:9093
-      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
+      # PLAINTEXT: for clients inside the compose network (kafka:9092)
+      # EXTERNAL: for clients on the host (localhost:9092, published below as 9092:9094)
+      KAFKA_LISTENERS: PLAINTEXT://:9092,EXTERNAL://:9094,CONTROLLER://:9093
+      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,EXTERNAL://localhost:9092
+      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
       KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
-      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
+      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT
...
     ports:
-      - "9092:9092"
+      - "9092:9094"

3. Make the mission-runner publish chatter to Kafka. By default, it keeps chatter in memory. Add this to your game-config/application.yaml, next to the existing game: section:

mp:
  messaging:
    outgoing:
      mission_chatter_publish:
        connector: smallrye-kafka

This switches the chatter channel from the default in-memory connector to a Kafka connector. Recreate the containers and play a mission against your local logic-service. Then check that chatter reaches the topic, bypassing your application:

docker compose exec kafka /opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mission-chatter

Keep this command at hand while you implement the consumer: events on the topic but not in your logs point at your application; no events point at the mission-runner or the broker.

Implementing a Kafka consumer¶

Spring for Apache Kafka integrates Kafka with Spring Boot. Keep the Spring Boot Kafka documentation at hand. Add these dependencies to your pom.xml (Spring Boot manages their versions):

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-kafka</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-kafka-test</artifactId>
    <scope>test</scope>
</dependency>

The second one is for the integration test later on. Work in small steps, so that a failure tells you where to look:

Step 1: Print the raw messages¶

Annotate a method of a Spring bean with @KafkaListener (see the reference), take the payload as a String, and log it. Start with these two settings in your application.yaml:

spring:
  kafka:
    bootstrap-servers: localhost:9092
    consumer:
      group-id: logic-service # fine on your own machine, see the warning on consumer groups

The bootstrap servers tell the client where the broker is. Spring Boot assumes localhost:9092 when you say nothing, so the listener would start without this line. It is the first setting that changes in every environment though, so state it explicitly. The consumer group is how the broker keeps track of what your service has read. Without one, your application refuses to start.

Make sure your local setup is running. When done correctly, you can play a mission against your local logic-service and watch the chatter scroll by. How many messages arrive per turn? Which of them would you want to store?

Step 2: Receive typed events¶

Replace the String with a MissionEvent. The messages are plain JSON produced by the game backend, not by a Spring application. Look at the JSON deserialization section of the Boot documentation: which deserializer, which default type? Spring Boot 4 uses Jackson 3, so look for the Jackson-prefixed deserializer; the older JsonDeserializer is deprecated. The backend is not a Spring application: what does that mean for the type headers? Check in your log that the polymorphic action and outcome come out as the right subtypes. The MissionEvent classes are provided by model-lib, so parsing itself is not yours to test.

Step 3: Process events in batches¶

Handling one event at a time means a potential DAO call per event, which is the load you want to protect your storage from. Spring Kafka can hand over everything the consumer fetched in one poll as a list: set batch = "true" on the @KafkaListener and take a List<MissionEvent> (see batch listeners). A batch is what accumulated while the previous one was being processed, up to max.poll.records events (500 by default). The busier the topic, or the slower your handling, the larger the batches. That is exactly when combining events pays off. Locally, with one mission and a turn every three seconds, most batches hold only a few events: your unit tests are where compaction gets proven.

A batch is a plain, ordered Java list, so filtering and compacting it needs nothing but ordinary streams or loops. Compaction means keeping the latest event per POI. Think about the key that identifies a POI, and look at the merge function of Collectors.toMap: which of two events about the same POI should survive?

Compact first, then interpret

Decide what an event means for the POI after compacting. If the last event about a deposit says it is empty, an earlier event of the same batch that said otherwise must not bring it back.

Events that arrive in different batches are not combined. That is acceptable: your DAO refreshes a POI it already knows from Lab 3.

Step 4: Hand the batch to your logic¶

Call the method you agreed on, and keep the listener itself thin: it knows about Kafka, and nothing else. Filtering and compaction belong to the logic, where a unit test can reach them without a broker.

Configuration¶

Make these settings configurable, with sensible defaults, using the approach from Lab 4: defaults in application.yaml, overridable through environment variables.

Setting	Default (local)	Why configurable
Bootstrap servers	`localhost:9092`	Differs per environment.
Topic	`mission-chatter`	Staging and production may use a different topic.
Consumer group	local use only	Identifies your service to the broker; see the note below.

@KafkaListener attributes accept property placeholders such as ${some.property:default}. Also look up the consumer's auto-offset-reset property and decide on the appropriate setting.

Consumer group and replicas

Consumers sharing a group split the events between them. Two instances with the same group ID each see only part of the chatter; two different groups each see everything. Which do you want for your service, and which for a shared broker with other teams' services?

A fixed name such as logic-service is a local-only default. On a shared broker, services of different teams with the same group would split the chatter between them, so every team needs its own group. Make sure staging and production can set a group that is unique to your team, and that the broker accepts (see staging).

Failures are normal

The game backend should only publish valid events. Even so, build in robustness: a backend that is newer than your model-lib may publish an event type you cannot read yet, and a consumer that stalls on one such message stops processing all chatter.

What happens when one message is not valid JSON? Try it: publish not json to the topic, type the line and press Enter, then watch your service.

docker compose exec kafka /opt/kafka/bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic mission-chatter

Read about the ErrorHandlingDeserializer and make sure one bad message cannot block the rest of the stream. In a batch, an unreadable message arrives as null: skip it, but make sure you notice it in your logs. And what if your own handler throws on a batch? Find out what Spring Kafka does by default (see error handling).

Experiment: decoupling in practice

Predict, then check: stop your logic-service (not Kafka) for a minute while the mission keeps running, then start it again. Which events does it process? Look at the size of the first batch after the restart, and at what your service does with it. Does that match your auto-offset-reset and active-mission decisions? (When does auto-offset-reset apply at all?) Also start it with a broker address where nothing runs: does it still answer the game backend's requests? Your Mission API must not depend on Kafka being up. After the restart, how does your service learn about the running mission again?

Assignment: consume the chatter

Add a configurable Kafka consumer for mission-chatter to your logic-service. It receives events in batches and ignores events for missions that are not active, for your own colony, and for events your logic has no use for. It compacts the rest, keeping the latest event per POI, and stores the deposits and hostile organisms it discovers as POIs through your Lab 3 DAO. Ten squad members mining one deposit cause one write, not ten. A malformed message does not stop the stream.

Testing the consumer¶

Test at the lowest level that can show the behaviour:

Test	Setup	Check
Filtering, compaction (`*Test`)	Plain JUnit, a list of events	What is kept, and how many writes a batch costs
Consumer (`*IT`)	`@SpringBootTest` with an embedded Kafka broker	A JSON message on the topic ends up as a POI

The MissionEvent classes come from model-lib, which staff maintain, so there is no separate test for parsing: if the JSON cannot be read, model-lib has to change. The integration test covers it anyway. Store the three examples as JSON files in your test resources, so the integration test can publish them.

Unit tests¶

Build lists of events (in Java, no JSON needed) and call your logic directly. Cover at least:

Filtering: an event that applies, and one that does not apply, for each reason you filter on.
Compaction: many events about one POI, in which the latest one wins; events about different POIs, which stay apart; a deposit emptied by the last event of a batch.

Make the writes observable: a stored POI would also be there after ten writes, so use a DAO that counts its save calls (extend or wrap your in-memory DAO) and assert on the number of calls.

Integration tests with an embedded broker¶

An integration test needs a Kafka broker. Running one in Docker next to your GitLab CI job would take extra setup, and you would need the same broker on every developer machine. The Kafka test starter avoids this: it provides an embedded Kafka broker that runs inside the test JVM. No Docker is involved, so the test works unchanged on your laptop and in your pipeline. Read how to test with embedded Kafka and use the @EmbeddedKafka annotation:

@SpringBootTest
@EmbeddedKafka(topics = "mission-chatter", bootstrapServersProperty = "spring.kafka.bootstrap-servers")
class MissionChatterIT { /* ... */ }

The bootstrapServersProperty attribute is how the test overrides your default localhost:9092: the property that points your application to the broker now points to the embedded one. Use a KafkaTemplate in the test to publish the raw JSON from your test resources, exactly like the producer in production does, then wait for the effect (for example with Awaitility, usually already on your test classpath through Spring Boot's test starters) instead of sleeping. Start the mission first: chatter of a mission that is not active is ignored.

Consider these questions while writing it:

Your test publishes a message right after the context starts. Could the consumer miss it? Why does auto-offset-reset matter here?
Which DAO implementation should this test use, and why does it not need a PostgreSQL service?
Does the test still pass if the topic name or consumer group is changed through configuration?
Publish a message that is not JSON, followed by a valid one. Does the valid one arrive?
Your other @SpringBootTest classes now start a listener too, against localhost:9092. Without a broker they still pass, but with your Compose broker running they join the consumer group of your running service and take its events. How do you keep your tests from interfering with a service running on the same machine? Tests can run under a consumer group of their own: put the setting in src/test/resources/application.properties, which Spring Boot loads next to your application.yaml.

The embedded broker prints ERROR lines while the test JVM shuts down (it removes its own log directory); if the tests pass, ignore them. Your existing CI test job should run this test without further infrastructure. Check that it does.

Assignment: test the consumer

Unit test the filtering and the compaction, with applies and does-not-apply cases. Add an integration test that publishes raw JSON to an embedded broker and verifies the resulting POI, and confirm it passes in your GitLab pipeline.

Operations¶

Your consumer needs a broker to talk to: locally, from the first implementation step on, and in every environment you deploy to. The local setup is already running; this section connects your service in staging and production.

Environment	Broker	Who publishes chatter	Authentication
Local	Docker Compose, `localhost`	Your local mission-runner	None
Staging	Shared broker, managed by staff	Your staging gamehost	SASL
Production	Shared broker, managed by staff	The competitive game backend	SASL

Connecting the staging environment¶

Staging uses a shared Kafka broker for all teams. Staff already created everything your group needs: look at your staging namespace.

kubectl get secrets -n group-N-staging
kubectl describe secret logic-kafka -n group-N-staging   # shows the keys, not the values

Item	Value
Bootstrap address	`devops-kafka-kafka-bootstrap.kafka.svc:9092` (reachable inside the cluster only)
Authentication	`SASL_PLAINTEXT` with mechanism `SCRAM-SHA-512`: a login, but no encryption
Topic	`group-N-staging-mission-chatter`: yours alone, one per group
Gamehost (publisher)	Secret `gamehost-kafka`: may write to your topic
Logic service (reader)	Secret `logic-kafka`: may read your topic
Consumer group	The broker only accepts groups starting with `group-N-logic-`

Both Secrets have the keys username, password and sasl.jaas.config. The last one is the complete login line a Kafka client expects, so you never assemble it yourself. Replace N with your group number everywhere.

Same broker, different rules

The Secrets are kept in sync with the broker by staff, so do not edit or recreate them. Locally you could use any consumer group; here a group outside your prefix is refused. Which setting from Configuration is affected?

Three things change:

Your logic-service. Point the settings you made configurable at this broker, and add the security settings. Any Kafka client property can be set through spring.kafka.properties.* (see the Boot documentation and Kafka's SASL documentation), and Spring Boot maps environment variables to properties: spring.kafka.properties.foo.bar is SPRING_KAFKA_PROPERTIES_FOO_BAR. Which three client properties does the broker require (hint: protocol, mechanism, and the login line)? Locally, none of them is set. An environment variable and a line in application.yaml are two spellings of the same property: compare the YAML from step 1 with the variable names above. Kafka's own property names contain dots (foo.bar): in YAML they stay one key under properties:.
Your Kubernetes manifests. Read the login line from logic-kafka into the Deployment's environment with secretKeyRef, as in Lab 5. Topic and consumer group differ per environment, and production will differ more: decide how your environment configuration handles this, and make sure the manifests still pass your validation. Environment variables are one way; Spring Boot also reads a config file from a ConfigMap (look up spring.config.additional-location, and mount the ConfigMap as a file with the same structure as your own application.yaml: only the values that differ). Neither restarts your pod when it changes: how do you make a changed ConfigMap or Secret take effect? Do this for staging first: production follows in the next section.
The staging mission-runner. Your staging gamehost must publish chatter to this broker, using gamehost-kafka, and it must run the newer mission-runner: the image tag 0.6.0, the same version as your local one. Both are settings of the gamehost chart, version 0.6.0 or newer: read the section Optional: enable Kafka chatter in helm show readme and check helm show values (as in Lab 4) for the names. Put them in your helm/gamehost-values.yaml and upgrade the release as in Lab 4. A gamehost upgrade resets the in-memory mission, and a changed Secret needs a restart of the pod.

Deploy through your pipeline, then check with Lab 4's diagnostics that the consumer starts, connects, and receives events from the staging gamehost. The first lines of your service's log show the client settings it started with; passwords are masked there. Three errors you may meet, and where each one points:

Log message contains	Look at
`SaslAuthenticationException`	The login line: wrong Secret, or a stale value.
`TopicAuthorizationException`	The topic name: it must be your own.
`GroupAuthorizationException`	The consumer group: it must start with `group-N-logic-`.

After such an error the listener, by default, stops for good, while the rest of your service looks healthy and keeps answering the game backend. Find the Spring Boot property that makes it retry instead (look in the spring.kafka.listener.* properties), so a fix on the broker side is picked up without a restart.

Connecting the production environment¶

In production, staff manage the game backend, which already publishes chatter to Kafka: you only configure your production logic-service. Production uses the same broker, but a different topic, which carries the chatter of the whole competition, not of your group alone.

Staff already created everything your group needs in production, as they did for staging:

Setting	Value
Broker, SASL	Same as staging
Topic	`mission-chatter` (shared by all groups and the backend)
Secret	`logic-kafka` in namespace `group-N` (same keys as in staging)
Consumer group	The broker only allows groups with the prefix `group-N-logic-`

Check the Secret with kubectl describe secret logic-kafka -n group-N, as before. The user in this Secret is a different one than in staging, and it cannot read your staging topic.

Use the same mechanism as for staging, with the production values: topic, consumer group and Secret. Never point production at the staging topic, or the reverse. Ship the change like any release: merge through staging, choose the next version following Lab 5's guidance, and tag it. Then repeat Lab 5's runtime checks. Do your logs show chatter arriving, and do chatter-discovered POIs show up in the POI API of your running service? Your own colony's events are on this topic too: do you ignore them?

Assignment: connect all environments

Make the consumer work locally with Docker Compose, in staging through your CI pipeline, and in production through a version tag. Credentials come from Secrets, never from Git. Show for each environment that chatter arrives and that your service keeps answering the game backend when Kafka is unavailable.

Practicalities¶

Lab 6 has no deadline of its own. Its deliverables are assessed together with a later submission; see the lab overview for all lab dates and deadlines.

Warning

All support outside of the physical Lab session is handled via GitLab work items (Plan > Work items). Learn more about the issue tracker and how to request support here. Do not expect support outside of the normal office hours (Monday to Friday, from 9 AM until 5PM).