Lab 4: Operating Your Service on Kubernetes¶

You have a tested service and a pipeline that publishes container images. Now make that service run somewhere other than your laptop: deploy it to Kubernetes, connect a standalone gamehost, and automate updates. Group A also moves its Lab 3 storage to PostgreSQL so recorded POIs survive a restart and can be read back. A third track adds endpoint security.

You can view the Lab 4 Slidedeck here.

Goals and scope¶

Declare, deploy, inspect, and debug a running application on Kubernetes.
Connect services inside the cluster and access them locally without public endpoints.
Deploy the verified commit image to staging from GitLab CI.
Group A: use PostgreSQL locally, in CI, and in staging; persist and read back POIs across restarts.
Authenticate gamehost calls and deliver credentials as configuration; Group A also gives POI readers a separate account and role.

Students	Required tracks
Group A	A: deployment, B: PostgreSQL, C: security
Group B	A and C; no PostgreSQL or Lab 3 DAO work required

Staging only

Use group-N-staging, replacing N with your team number. Do not deploy to group-N (production). Lab 5 introduces production and release deployment.

What does Kubernetes do?¶

Docker gave you a way to package and run a container. But who starts it on a server, replaces it when it fails, connects it to other services, and deploys the next version?

Kubernetes is a container orchestration platform. You declare a desired state, such as "run one instance of this image", through its API. Controllers continuously compare that with what is actually running and act to bring the two into agreement.

A cluster contains a control plane, which coordinates this work, and worker nodes, which run the containers. Staff operate the cluster; you manage workloads in your team's namespace. A namespace groups related resources and scopes access policies; it is not a separate VM.

Start with Kubernetes Simply Explained — TechWorld with Nana and the Kubernetes overview. Keep these resource names nearby as you work:

Resource	Its role in this lab
Pod	Runs containers together. A replacement pod can have a different IP address.
Deployment	Manages ReplicaSets; changing its pod template triggers replacement.
ReplicaSet	Keeps the requested number of matching pods running.
Service	A stable network address for pods selected by labels.
ConfigMap	Delivers non-sensitive configuration to a workload.
Secret	Delivers credentials to a workload without embedding them in its image.
StatefulSet	Manages database pods with stable names and storage associations.
PersistentVolumeClaim (PVC)	Requests storage that outlives an individual pod.

Kubernetes does not build your image, repair application bugs, or make a database into a backup. Those remain separate responsibilities.

Before splitting the work¶

Complete cluster onboarding on each teammate's machine. Check that you can list pods in staging, even if the list is empty.
In your logic-service repository, add Helm through mise:

mise use helm@4.2.3
helm version --short

Commit your project's mise.toml with pinned tool versions; teammates install them with mise install. If mise is not set up yet, follow development environment setup.

Confirm Lab 2 publishes a usable commit.<short-sha> image, not just latest. GitLab's $CI_COMMIT_SHORT_SHA has eight characters; a local git rev-parse --short abbreviation may differ. Copy the actual tag from the registry or pipeline. After merging, staging runs the default-branch pipeline's commit, which may be a merge commit rather than your feature branch's last commit.
Create work items under the combined Lab 3–5 milestone. Agree on the contracts below and a safe way to share credentials before working in parallel.

Shared contract	Value
Logic Deployment and container name	`logic-service`
Logic Service	`logic-service`, port `80` → application port `8080`
Gamehost's logic URL	`http://logic-service`
Group A database profile / database Service	`postgres` / `postgres-service:5432`
API Secret / keys	`logic-api-credentials` / `GAMEHOST_USERNAME`, `GAMEHOST_PASSWORD`; Group A also `POI_USERNAME`, `POI_PASSWORD`
Configuration and credentials	Non-secret settings in Git; staging credentials outside Git

Split by track or use peer programming. Working together on an unfamiliar part is just as valid as dividing the tracks. Track A can deploy the existing H2-backed image while Track B develops locally; Track C can also be developed independently on its branch.

Before merging authentication changes into main, agree on the credentials/configuration and coordinate the matching gamehost and HTTP-test changes. Review and integrate early; rotate the cluster driver so everyone operates and diagnoses the system.

Three-hour milestone for the physical lab session: aim for the logic service and gamehost running manually in staging. In parallel, Group A should have local PostgreSQL running and an existing integration test passing against it; the security track should have gamehost authentication working locally (and Group A's role separation checked). Finish POI read-back, CI automation, and combined staging checks after the session, before the shared checkpoint.

Track A: Deploy and operate the staging stack¶

A1. Explore your namespace¶

Check your target before changing anything:

kubectl config current-context
kubectl config view --minify
kubectl get pods,services -n group-N-staging

Your context should be group-N-staging. Keep using the explicit namespace flag while learning.

A2. Deploy the logic service manually¶

Create k8s/deployment.yaml and k8s/service.yaml. Keep Helm values in a separate helm/ directory, so applying k8s/ cannot mistake a values file for a Kubernetes resource.

Your manifests must:

Run a published commit.<short-sha> image on port 8080, initially with the existing H2 setup. Group B keeps its existing storage configuration.
Use the Deployment, container, and Service names from the shared contract.
Match Service selectors to pod labels; use a ClusterIP Service, not a public route.
Use one replica and the Recreate strategy for now: brief downtime is acceptable, and two processes must not independently own your in-memory mission state.

Use the namespace's existing resource defaults; resource tuning and health probes come in later labs.

For the private logic-service image, create a project Deploy Token with read_registry scope under Settings → Repository → Deploy tokens; see GitLab's instructions. Copy both the generated username (usually gitlab+deploy-token-<n>, not your own username) and the token, which is shown only once.

Create a registry Secret named gitlab-credentials for gitlab.stud.atlantis.ugent.be:5050, and reference it through imagePullSecrets. Start with Kubernetes' private-registry instructions. If the token expires, create a replacement and update the Secret. Keep tokens out of Git and shell history. The course gamehost image is public and does not need this token.

Start from the Deployment example and Service documentation, then adapt them. Use kubectl explain deployment.spec.template.spec.containers to investigate fields.

kubectl apply -f k8s/ -n group-N-staging
kubectl rollout status deployment/logic-service -n group-N-staging --timeout=120s

A Running pod or completed rollout is not proof of a working API: without readiness probes, Spring Boot may still be starting. Allow a little startup time and retry; inspect logs if it continues failing.

Next you can run a port-forward to access the kubernetes service from your laptop. The Service exposes port 80, which mapped to the container's application port 8080. The following command forwards your local port 8081 to the Service's port 80.

kubectl port-forward service/logic-service 8081:80 -n group-N-staging

Keep the forward running and use your existing HTTP requests against http://localhost:8081. Group A can reuse its Lab 3 Postman collection. You could even run a local gamehost against the staging logic service if you set up ports and URLs correctly.

A3. Install the standalone gamehost with Helm¶

Helm is a package manager for Kubernetes, commonly used to distribute applications to others. A chart packages the related resources; a named installation is a release, and a values file supplies settings for that installation.

We distribute the gamehost as a chart so you can install it without writing and maintaining all its manifests yourself. The chart manages its Deployment, Service, and ConfigMap. You still write your own logic-service manifests to learn how those resources fit together.

Read the Helm quickstart for the install/upgrade workflow and chart concepts for how charts and values fit together.

Inspect the published chart instead of guessing its configuration:

helm show readme oci://gitlab.stud.atlantis.ugent.be:5050/devops/images/charts/gamehost --version 0.4.15
helm show values oci://gitlab.stud.atlantis.ugent.be:5050/devops/images/charts/gamehost --version 0.4.15

Tip: if you have trouble reading this in your terminal, pipe it to less or redirect to a file so you can open it in an editor. The chart's README explains the standalone mode and its available configuration options. The values file shows the default settings, which you can override in your own helm/gamehost-values.yaml.

Create helm/gamehost-values.yaml, starting with your colony and a built-in opponent:

game:
  colonies:
    - id: group-N
      logic-url: http://logic-service
    - id: bot

Replace N. http://logic-service uses port 80, as exposed by your Service, not the chart README's example port 8080. The URL is resolved inside the cluster: localhost would refer to the gamehost pod itself, not your laptop or logic pod.

helm upgrade --install mission \
  oci://gitlab.stud.atlantis.ugent.be:5050/devops/images/charts/gamehost \
  --version 0.4.15 --namespace group-N-staging \
  --values helm/gamehost-values.yaml --wait --timeout 5m
kubectl port-forward service/mission-gamehost 8090:8080 -n group-N-staging

Open http://localhost:8090/_ui/. Keep the forward bound to localhost: this standalone UI has no login. Verify a mission actually calls your logic service using both services' logs, not just that the UI loads. Gamehost upgrades reset its in-memory mission; restart the port-forward if it exits.

A4. Diagnose, don't guess¶

Use this small toolkit on the resources you just deployed. Ensure either correct staging kubeconfig context or add -n group-N-staging:

Question	Starting command
What exists, and which pod is replacing which?	`kubectl get deployments,replicasets,pods -o wide`
Why did scheduling or startup fail?	`kubectl describe pod <pod-name>`
What did the app report?	`kubectl logs deployment/logic-service --tail=100`
Does the YAML on the clsuter match what you applied?	`kubectl get deployment logic-service -o yaml`
What did the previous crashed container report?	`kubectl logs <pod-name> --previous`
Which pods does the Service select, and are they marked ready?	`kubectl get endpointslices -l kubernetes.io/service-name=logic-service -o yaml`
What happened recently?	`kubectl get events --sort-by=.lastTimestamp`
How do I run an available command inside a container?	`kubectl exec <pod-name> -- <command>`

Images do not necessarily contain a shell, curl, or debugging tools. An absent command is not evidence that networking failed. See debugging running pods and debugging Services.

Set up completion for your frequently used CLI tools

Most CLI tools offer shell completion, meaning they can suggest and complete commands and resource names. This is especially useful for kubectl and helm: instead of typing / copying the exact pod ID, you can type the first few characters and press TAB to complete it. This is particularly helpful when you have multiple pods with similar names.

See kubectl completion for instructions.

Exercise: introduce one reversible fault, such as an invalid image tag or Service selector. Predict the symptom, collect evidence, correct the YAML, reapply it, and verify recovery. Keep a separate terminal open with kubectl get events --watch to observe the cluster's reactions. What events happen in succession when you restart a Deployment, or delete a pod?

Optional tools: k9s (mise use k9s) offers a terminal UI; Headlamp and Kubernetes integrations in VS Code/IntelliJ offer graphical views. They are optional, not substitutes for understanding the commands and resource relationships.

A5. Automate staging deployment¶

Extend the Lab 2 pipeline with a deploy stage after publish. CI connects through the shared GitLab Kubernetes agent, not a student's SSO kubeconfig. Start with GitLab's agent access documentation.

Your job must:

Run only for commits on the default branch (main), not feature branches, merge-request pipelines, or Git-tag pipelines. Production deployment from tags comes in Lab 5.
Use gitlab.stud.atlantis.ugent.be:5050/devops/images/k8s:1.36.5 as the job image: this course mirror of alpine/k8s already contains kubectl and a shell. Select devops/infra:k8s-agent with kubectl config use-context. See CI additions for the image settings.
Apply the logic-service resources to group-N-staging, using exactly $CI_REGISTRY_IMAGE:commit.$CI_COMMIT_SHORT_SHA from that pipeline's publish job.
Wait for rollout completion with a timeout and fail if it does not complete.

Render/substitute the intended image before applying the Deployment, rather than first deploying an old checked-in image. Investigate kubectl set image --help, especially --local and -o yaml, for one starting point. Use job rules to restrict deployment. Keep manifests and CI readable; do not commit a kubeconfig or print credentials in job logs.

This CI job deploys only your logic-service Deployment and Service. Install and update the gamehost (and Group A's PostgreSQL) manually with Helm, and create the required Secrets separately. Do not add Helm installations or Secret creation to this job; complete those setup steps before relying on automated logic-service deployment.

Push a change through an MR and merge it. Verify the pipeline and the running pod's image agree. Observe that the MR pipeline has no staging job. Check the tag case with pipeline validation/simulation, not by pushing a new tag: a tag pipeline also publishes latest.

Track B: Move to PostgreSQL (Group A)¶

Group B skips this track. Start from your Lab 3 implementation: the DAO interface, SQL DAO, entities, tests, and POI API already exist. Do not replace them with a different example application. The linked PostgreSQL setup and inspection guide supplies infrastructure examples; your existing DAO remains the boundary between logic and storage.

B1. Keep H2, add a PostgreSQL profile¶

Add the PostgreSQL JDBC driver and application-postgres.yaml. With no profile, the service must still use H2; activating postgres selects the PostgreSQL datasource, not a new DAO. The memory profile from Lab 3 still selects the in-memory DAO, a separate choice.

Start PostgreSQL 18.6 with Compose and a named volume; add pgAdmin for inspection.
Configure URL, username, password, and driver for the postgres profile. Use DB_URL, DB_USER, and DB_PASSWORD so local, CI, and staging settings can differ without rebuilding.
Choose an explicit schema setting: ddl-auto: update is acceptable for this learning setup; create-drop would defeat persistence. Proper versioned schema migrations are not required here.
Inspect the actual tables and enum values in pgAdmin. Fix mapping/query differences without changing your DAO contract or moving SQL into the decision logic.

Starting points: local setup, Spring Boot profiles and database initialization.

B2. Read back stored POIs¶

Keeping rows is only useful if your logic reads them again. After a service restart, the SQL DAO must still return the previously stored POIs for a mission. If your logic keeps a working set, populate it from the DAO when that mission is initialised, rather than starting empty. A repeated mission-start notification must not erase the stored sightings.

Test this with fresh logic/DAO state against the same database, not just an unchanged Java object. For the restart check, you may re-send POST /missions with the same mission ID before reading its POIs through the API. The backend can also reinitialise a forgotten mission; see the Mission API contract. Keep Lab 3's mission-end POI cleanup.

This does not fully restore mission state. Your mission registry, latest turn data, and other in-memory decisions may still be lost. A POI endpoint returning 404 until the mission is reinitialised is therefore expected; full mission recovery is not required.

Optional extension: persist mission data or any other state you want to keep, such as strategic targets. Start by listing the state your decisions need after a restart, then introduce a DAO and your own persistence classes for it, following the layering from Lab 3. The model-lib records themselves are not JPA entities.

B3. Test against real PostgreSQL in CI¶

Keep the default H2 verification job. Add a separate required PostgreSQL integration-test job, and make publishing wait for both. Run the existing SQL DAO contract and HTTP integration tests against PostgreSQL, including the POI read-back checks; do not rewrite them as mocked repository tests.

Start with a GitLab PostgreSQL service and the test command and connection checks. Testcontainers is an alternative if your runner supports its Docker requirements. H2's PostgreSQL compatibility mode is not a substitute.

Prove which database the job used. Add an integration-test assertion about the database product reported by an actual connection, not just the profile name or configured URL. It must run in the PostgreSQL job even if the profile is wrong. Confirm that job fails with an invalid database URL and, separately, a missing/wrong PostgreSQL profile; restore both afterward. The companion guide gives a metadata starting point.

B4. Deploy the database and integrate with Track A¶

Follow cluster PostgreSQL setup: CloudPirates chart 0.21.1, PostgreSQL 18.6, a 2Gi PVC on k8s-studstorage, and no public database endpoint. Create the admin and application Secrets manually; keep actual values out of Git and Helm values.

Update the logic Deployment to activate postgres, with DB_URL=jdbc:postgresql://postgres-service:5432/colonyops. Read DB_USER and DB_PASSWORD from logic-db-credentials using secretKeyRef, keys CUSTOM_USER and CUSTOM_PASSWORD. Use the application account, not the PostgreSQL superuser.

Before merging this Deployment change, create the Secrets and wait for PostgreSQL to be ready. Commit the profile and environment references in your manifest: CI reapplies the checked-in configuration, so manual-only changes can be undone by the next pipeline—even silently returning the app to H2. Missing dependencies instead cause startup/rollout failures.

After deployment:

Check that the startup log lists postgres as an active profile and inspect staging rows through pgAdmin. Running alone is not proof of the profile or database connection.
Use Postman to start a test mission and record POIs. Restart only the logic service, check that rows remain, then re-send the start request with the same ID and verify POI read-back. Keep this fixture separate from the gamehost's automatically ending missions.
Replace the database pod and check that data and its bound PVC survive. Neither replacement test demonstrates a backup.

Track C: Secure the REST endpoints¶

Anything that can reach your service can currently drive missions and, for Group A, read back recorded POIs. Close that with HTTP Basic authentication.

Both groups protect the Mission API with a gamehost account and role. Group A also has a second caller: you query points of interest with a separate account and role, so neither caller can use the other's endpoints. Group B does not add POI endpoints or a POI viewer account.

Plan before you implement! Discuss as a team and design the code before you start implementing.

C1. Accounts and configuration¶

Use these role names and credential contracts so your service, tests, gamehost configuration and staging Secret agree:

Caller	Endpoints	Role	Username	Password
Gamehost	your Mission API	`GAMEHOST`	your colony ID	that colony's secret
You, through Postman or `curl` (Group A only)	your POI endpoints	`POI_VIEWER`	your choice	your choice

Group A chooses its POI_VIEWER account. The gamehost account is different: its username is the colony ID, and for the competition in Lab 5 its password is provisioned for you.

Read GAMEHOST_USERNAME and GAMEHOST_PASSWORD from the environment; Group A also reads POI_USERNAME and POI_PASSWORD. This lets local, CI and staging differ without rebuilding the image. Bind them in application.yaml through ${VARIABLE:default} placeholders, so the service still starts locally without anyone exporting anything; Spring Boot's externalised configuration explains how those resolve and in what order sources win. A default is a convenience for local development; no credential that protects anything real belongs in Git.

With a secret configured, the gamehost authenticates as the colony it is calling for: every Mission API request carries the colony ID as the username and that colony's secret as the password. Give the colony that carries your logic-url a secret in your local game server configuration:

game:
  colonies:
    - id: colony-1 # (1)!
      logic-url: http://host.docker.internal:8080
      secret: local-development-only-password # (2)!
    - id: colony-2

The username half of the gamehost account, so GAMEHOST_USERNAME must be colony-1 here. The username is the colony ID, not a name you pick.
The password half, so this must equal GAMEHOST_PASSWORD. A colony with no secret sends no credentials at all. Your unsecured service currently accepts those requests.

Authentication is not encryption

Basic auth base64-encodes the password, which is encoding, not encryption: whatever can read the traffic can read the credentials. Plain HTTP between pods is acceptable for this lab, but it means the password is protected by the network, not by the scheme.

C2. Configure Spring Security¶

Add spring-boot-starter-security as a dependency in your pom.xml.

One @Configuration class carries the security setup, and its two @Enable... annotations each switch on a separate mechanism:

@Configuration
@EnableWebSecurity // (1)!
@EnableMethodSecurity // (2)!
class SecurityConfig {

    @Bean
    SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
        return http
                /* your security decisions */ // (3)!
                .build();
    }

    @Bean
    PasswordEncoder passwordEncoder() { /* ... */ } // (4)!

    @Bean
    UserDetailsService userDetailsService(/* ... */ ) { // (5)!
      /* ... */ // (6)!
    }
}

Imports the configuration that builds the filter chain and supplies the HttpSecurity injected below. Spring Boot applies it for you in a servlet web application, but keep it explicit: a test context that is not one does not get Boot's version, and the bean below then has nothing to build from.
Switches on the method annotations of C3. Without it they are inert, and an inert annotation looks exactly like a working one until something calls the endpoint.
The calls listed below. Each returns http again, so they chain.
Your configured passwords arrive as plain text, and the running service should be comparing hashes (Bcrypt). You can customize the PasswordEncoder implementation used by Spring Security by exposing a PasswordEncoder Bean. (password storage).
Convert your injected env variables to a POJO (see below). You can inject this class and the PasswordEncoder here.
Reject blank credential values and, for Group A, identical account usernames at startup without logging passwords. Register the account(s) with encoded passwords: In-memory authentication.

The security configuration requires some @Beans to achieve our goals. The SecurityFilterChain is where you define your security decisions.

Each of the method calls on the http variable is a decision, not a line to copy. Work out what the default does first, then what your API actually needs:

csrf is on by default to protect against browsers submitting unintended requests with automatically attached credentials. This can include Basic credentials, not just session cookies: statelessness alone is not protection. Decide what this lab's non-browser API clients need, and record why your configuration lets them send POST/PUT/DELETE requests. Cross Site Request Forgery
sessionManagement decides whether a successful login is remembered in an HTTP session. The gamehost sends Basic credentials on every request and does not reuse a session cookie. What session policy fits that client? Session management
headers sets the security headers on every response. One of them tells browsers not to render your pages inside a frame, and the H2 console is built out of frames, so this matters if you use the local H2 console (Lab 3). Keep console access a local-development exception, not an endpoint you enable in staging. Security HTTP response headers
authorizeHttpRequests says which requests may skip authentication. A local H2 console is one candidate. Spring also authorises internal error dispatches; ensure existing API errors still return their intended status rather than assuming /error must always be public. Your API endpoints should require a caller you can name. Authorize HttpServletRequests
httpBasic enables the scheme itself. Its realm name is what every 401 returns in its WWW-Authenticate header, which is how a client knows what to retry with; give it something recognisable instead of the default. Basic authentication

Getting ENV-variables into your code as a Plain Old Java Object (POJO)

Registering ENV-variables in the application.yaml placeholders land the values in properties you name yourself; getting them from there into your code is a second step. One way is to manually inject every variable with the @Value annotation above variables.

@ConfigurationProperties(prefix = "...") binds a whole group of related properties onto one single object (POJO). A record also works well, and its components are filled through the constructor. Much cleaner instead of scattering @Value across every class that needs a credential. You end up with one type that names your credential values and one thing to inject, and Spring reports a missing or unconvertible value once, at startup. This system has to be switched on, with @EnableConfigurationProperties on the configuration that uses it (or by scanning for it, but we use the first option): type-safe configuration properties.

C3. Put the rule on each endpoint¶

Who may call what is declared on the endpoint method, not as URL patterns in a central list. Both groups use GAMEHOST on every Mission API method; Group A uses POI_VIEWER on POI methods:

@PostMapping
@PreAuthorize("hasRole('GAMEHOST')")
public ResponseEntity<Void> startMission(@RequestBody Mission mission) { ... }

Extracting Security rules to their own location

This role name appears in different locations in your codebase, and nothing checks them against each other, making this prone to a typo. Decide where that string should live so there is exactly one of it, before you write the second copy. An annotation attribute has to be a compile-time constant, which rules some answers out and makes one of them natural. We suggest a dedicated constants class. The example includes Group A's second role; Group B only needs the gamehost constants.

public final class ApiRoles {

  public static final String GAMEHOST = "GAMEHOST";
  public static final String POI_VIEWER = "POI_VIEWER";

  public static final String GAMEHOST_ONLY = "hasRole('" + GAMEHOST + "')";
  public static final String POI_VIEWER_ONLY = "hasRole('" + POI_VIEWER + "')";

  private ApiRoles() { // (1)!
  }
}

A private constructor makes it so no new ApiRoles object can be instantiated, as this is not the point of this class.

Every endpoint method needs such a rule, including the ones you add later: a method without one is reachable by any authenticated account, which is a decision you did not make. For Group A, a caller authenticated as one role must be refused on the other's endpoints, and that refusal is a 403, not a 401 — the service knows who is calling and still says no. Demanding credentials at all stays the filter chain's job; who may do what is the method's.

For Group A, method rules let POI endpoints nested under /missions/... have their own role; a broad /missions/** URL rule could accidentally include them. Two things to watch: the annotations do nothing until method security is enabled, and they guard calls arriving from outside the bean — which is every HTTP request, but not a call a bean makes to its own method. More in method security.

That string is SpEL

The rule is written in the Spring Expression Language: a string parsed and evaluated per request against a context holding the authenticated caller, which is how hasRole reaches your credentials from inside an annotation. Two consequences. Nothing checks that string at compile time, so a typo can break authorisation at runtime rather than fail the build. And hasRole('GAMEHOST') asks for the authority ROLE_GAMEHOST, where hasAuthority('GAMEHOST') asks for GAMEHOST exactly, so your expressions and the roles you give your accounts have to agree on which of the two they mean. The language reference covers the rest; you need very little of it here.

C4. Prove it locally¶

Use gamehost 0.4.15 for these checks. If you still use an older container, follow choosing and updating the gamehost image; editing its configuration does not update the image.

Restart your service and the gamehost, and check the pair rather than one side:

Verify successful Mission API calls, not just a loading UI or continuing turns: the gamehost can keep running after logic errors. Its error message may omit the HTTP status.
curl -i without credentials returns 401 with a WWW-Authenticate header; the same request with curl -u and the gamehost account returns the normal response.
Group A: the POI account is refused on a Mission API endpoint, and the gamehost account on a POI endpoint (both 403).
Change the secret on one side only, find where that surfaces, and put it back.

The logic service need not log refusals at its default level. Temporarily set logging.level.org.springframework.security=DEBUG (or environment variable LOGGING_LEVEL_ORG_SPRINGFRAMEWORK_SECURITY=DEBUG) and reproduce a single failed request. Use the security trace together with curl -i to distinguish authentication/authorisation failures from connection failures. Remove the logging override afterward.

Safe logging (optional)

No custom logger is required. If you add one, log the outcome, method and path, never passwords or Authorization headers. Inspect and redact diagnostic output before sharing it; see the OWASP logging guidance.

Your existing integration tests now drive an API that rejects them, so authenticate their clients too. A class that only calls Mission API endpoints can configure one client in @BeforeEach, without changing each request. With JUnit's default per-method test lifecycle, each test gets a fresh instance:

@SpringBootTest(
        webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT,
        properties = {"<your username property>=it-colony", "<your password property>=it-secret"}) // (1)!
@AutoConfigureRestTestClient
class MissionApiIT {

    @Autowired
    private RestTestClient client;

    @BeforeEach // (2)!
    void authenticateAsTheGameBackend() {
        RestTestClient.Builder<?> builder = client.mutate(); // (3)!
        builder.defaultHeaders(headers -> headers.setBasicAuth("it-colony", "it-secret"));
        client = builder.build();
    }
}

Adds settings to this test's environment, at a higher precedence than application.yaml, so the test pins the credentials it authenticates with instead of inheriting whatever your local development defaults happen to be. More on testing Spring Boot applications.
Runs before every @Test in the class, on a fresh instance of it. More on @BeforeEach.
mutate() copies the injected client, and defaultHeaders puts the credentials on every request that copy makes. A Group A class that starts missions and reads POIs builds two clients this way and changes its requests to use the appropriate one.

One test profile passes, another returns 401

If a subclass reruns your tests with @ActiveProfiles("memory"), it inherits @SpringBootTest. Redeclaring that annotation can replace the parent's credential properties. Remove a redundant declaration, or explicitly preserve the settings the subclass needs.

For every protected endpoint, test missing credentials (401 with WWW-Authenticate), an incorrect password (401), and the correct account (the endpoint's normal response). Group A also tests the other account (403). Test behaviour: annotations being present does not prove method security is enabled. Re-export your Postman collection using disposable local/test credential variables, not staging passwords or credentials typed into every request.

C5. Deliver the credentials in staging¶

The image must not contain staging credentials. Use the shared Secret creation and inspection workflow to create logic-api-credentials in group-N-staging:

Secret key / application environment variable	Value
`GAMEHOST_USERNAME`	`group-N`, exactly the colony ID in your gamehost values
`GAMEHOST_PASSWORD`	A generated staging password
`POI_USERNAME` (Group A only)	A different username of your choice
`POI_PASSWORD` (Group A only)	A different generated staging password

Read each applicable key into its matching environment variable with secretKeyRef. Commit these references in the Deployment: CI reapplies that configuration. Never commit the Secret values or put them in Helm values or CI logs. Start with Kubernetes' Secret-backed environment variables.

Update helm/gamehost-values.yaml so the staging gamehost reads the same password:

game:
  colonies:
    - id: group-N
      logic-url: http://logic-service
      secret-ref:
        name: logic-api-credentials
        key: GAMEHOST_PASSWORD
    - id: bot

Use secret-ref, not an inline staging secret. Helm replaces lists rather than merging them, so retain your full colonies list and any other settings. The chart reads only the referenced password key; the colony ID supplies the username.

Integrate in this order: create the Secret, upgrade gamehost with the A3 command (chart 0.4.15), then merge/deploy the secured logic service and its environment references. An unsecured service can accept the gamehost's extra header; securing it first causes refusals until gamehost is configured. The Helm upgrade resets the in-memory mission; restart the port-forward if needed.

If you experimented with a literal environment value using kubectl set env or an edit, remove that override before restoring its Secret reference. For example, in staging:

kubectl set env deployment/logic-service GAMEHOST_PASSWORD- -n group-N-staging
kubectl apply -f k8s/ -n group-N-staging

Use your normal rendered staging manifests if they need image/configuration substitution. An apply error saying valueFrom ... may not be specified when value is not empty means the live entry still has a conflicting literal value; schema validation alone cannot detect this live-state conflict. Restore the declarative configuration before relying on CI again.

Through the A2 port-forward, verify missing/wrong credentials give 401, your account(s) behave as locally, and no password appears in logs. Also verify successful calls from the in-cluster gamehost; use C4's temporary DEBUG logging if default logs are inconclusive. Then change the gamehost password in the Secret: work out what must happen for both logic and gamehost to use it, verify your answer, and restore successful calls.

Combined Lab 3–5 checkpoint¶

There is no separate Lab 4 report. Add concise evidence to the combined submission due 3 November 2026 using the template below. The practical requirements and checks in the tracks still apply; you do not need to reproduce all their results in the report.

Lab 4 report section template¶

Paste this section into your Checkpoint 2 Report, not a separate work item. Fill in the first three sections; Group A also completes the fourth. Keep answers short and specific, and link to evidence instead of collecting screenshots or copying full logs. Full mission recovery remains optional: the last question asks for a design proposal, not an implementation.

Checkpoint 2 Report — Lab 4 section

## Lab 4: Operating in staging

<!-- Keep answers short and specific: a few sentences or bullets per section.
Link to relevant evidence rather than reproducing logs or configuration.
Never include passwords, tokens, or Authorization headers.
Delete the italic prompts once answered. -->

### 1. Diagnosing a failure

- **Symptom:** _Describe one fault you investigated, either encountered naturally or deliberately introduced. What did you observe?_
- **Diagnosis:** _What was the cause? Include one decisive observation or short redacted excerpt, and explain how it helped identify the cause._
- **Recovery:** _What did you change, and how did you verify that the affected behaviour worked again?_

### 2. From pipeline to running service

- **Deployment pipeline:** _Link one successful default-branch deployment pipeline._
- **Running image:** _Identify the exact image running in staging. How did you establish that it was the image published by this pipeline?_
- **Verification:** _What does a completed Kubernetes rollout prove? Describe the separate check you used to establish that the deployed application worked._

### 3. Security decisions

- **Sessions and CSRF:** _Explain your session policy and CSRF configuration in relation to the clients that call your API. Why do these choices fit those clients?_
- **Behavioural evidence:** _Link one focused test or give one request/result that supports your explanation. State what it demonstrates._
- **Remaining risk:** _What risk does Basic authentication over plain HTTP leave unresolved?_

### 4. Persistence and recovery — Group A only

<!-- Group B: remove this section. -->

- **Restart observation:** _After restarting only the logic service, what remained available and what needed reinitialisation? Give one concrete POI read-back observation._
- **Limits:** _Why does this observation not demonstrate full mission recovery or a database backup?_
- **Recovery proposal:** _Do you plan to add full mission recovery? Why or why not? Either way, briefly outline how you would approach it: what additional state must survive, how it would be stored and restored, and how you would test recovery._

<!-- Full mission recovery is optional. This question asks for a design proposal,
not an implementation. -->

Checklist¶

For support outside the session, use GitLab work items. Include what you expected, what happened, relevant redacted output, and what you already tried. Do not delete database PVCs or uninstall PostgreSQL as routine cleanup.