Using go private packages on local and with pipelines

Let's start with outlining the motivation:

• you want to extract some of your go code into an independent package
• you want to have this package on github or other SCM
• at some point you want to use this package in multiple projects
• you want to keep this package private

In my case, I use GitHub, and I deploy to self managed VPS using Docker, through amazing Coolify platform.

Let's assume your GitHub handle is pickman42, and your private package is named common.

GitHub setup

Create new private access token on GitHub. Go to user → settings → developer setting → personal access tokens and create a classic token:

• create new token (classic)
• select repo permission category
• copy token

---

Local setup

To your ~/.netrc file, add:

machine [github.com](<http://github.com/>) login pickman42 password TOKEN

where TOKEN is the token you copied from GitHub.

To your ~/.zshrc file, add:

# private repos pattern for using private go modules from github
export GOPRIVATE=github.com/pickman42
# access token for docker builds of go webapps
export GITHUB_ACCESS_TOKEN=TOKEN

where TOKEN is the token you copied from GitHub.

---

Local usage

install common package to a new project

go get github.com/pickman42/common

use local version of the package

For example when you need to do changes there:

# clone the repo to the project root
git clone https://github.com/pickman42/common.git

# either init workspace for the first time (or if go.work are removed)
go work init common .
# or rename back from setup for docker build
mv go.work.local go.work
mv go.work.sum.local go.work.sum

push new changes and release

Once done with local changes to common package, use git push from the common folder.

Then, make a release on github, to get the version tag (eg. v1.2.3)

On the project repo, run go mod tidy to switch to this new release.

build Docker image locally

using common package from github

# to install common package properly if not installed yet
go mod tidy

# to skip local version copy
mv go.work go.work.local
mv go.work.sum go.work.sum.local

# to build docker image and pull common from github
docker build . --build-arg GITHUB_ACCESS_TOKEN=$GITHUB_ACCESS_TOKEN

using local common package (to verify current changes in common package)

# leave go.work files as they are, from the setup to use local common package

# in Dockerfile, comment out following line:
RUN rm -rf /app/common

These cover necessary workflows I run into on local. The Docker part is helpful for testing the build locally.

---

CD pipeline usage (Coolify)

1. add environment variable to the pipeline runner

Ideally create a different access token for each environment (prod/test/dev)

GITHUB_ACCESS_TOKEN=TOKEN

where TOKEN is the token you copied from GitHub.

2. update Dockerfile

For your Go build, either in standalone or multi-stage

FROM golang:1.23 AS backend

# ... necessary setup

# remove local workspace copy when building image locally
# ci/cd environment build gets the package from private repo on github and the directory does not exist
RUN rm -rf /app/common

# setup private module pattern and github access token
ARG GITHUB_ACCESS_TOKEN
ENV GOPRIVATE='github.com/pickman42'
RUN echo "machine github.com login pickman42 password ${GITHUB_ACCESS_TOKEN}" > ~/.netrc

RUN make install
RUN make build

3. update docker-compose

Add GITHUB_ACCESS_TOKEN build argument to your backend container configuration

services:
  backend:
    build:
      context: .
      args:
        - SOURCE_COMMIT
        - GITHUB_ACCESS_TOKEN