Go E2E tutorial Part 1: Clean Architecture and Folder Structure

In this part we will be discussing about the of pattern how the files and folders are structured in order to achieve clean and SOLID architecture.

You might have heard about this blog from 2012: https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html

If I may simplify the points, here are the requirements to achieve clean architecture:

1. Independent of Frameworks. The architecture does not depend on libraries or features.
2. Testable. The business rules can be tested without the UI, Database, Web Server, or any other external element.
3. Independent of UI.
4. Independent of Database. You can swap out SQL Server, for Mongo.
5. Independent of any external agency.

But it was still abstract to me. Until I saw diagram in this post: https://github.com/bxcodec/go-clean-arch

Now that makes things much clearer. It basically says that, if Service wants to talk with Controller, or wants to talk to Repository, it may not directly to the implementation, but must be via an interface. Therefore changes of controller or storage doesn’t need to be followed by change of business logic implementation. Somehow we can relate this to SOLID principle, at least to me the letter S,L and D letters are tightly coupled with clean architecture requirements. When we put the interface into picture, more or less it may look like this:

But wait, Domain/Model does not need interface?

Well they don’t need interface to be used by Repository, or Service, or Controller, because they by default must be already interfaces.

One thing that probably I don’t quite agree with the diagram is on the Microservices. To me it is not fit under Repository, but should be in Controller/Delivery. Because when we are talking about machine-to-machine communication in Go, usually they are in form of gRPC, REST API, or via Twirp RPC. I do understand the author’s idea for Controller is basically about incoming request. But to make the pattern consistent, it is possible to create new category of “outgoing” or “external” under the Controller. More of this example in other part of tutorial.

Folder Structure

Alright, I would like to discuss about Domain/Model/Entity first, but before that let’s dig dive on what the folder structure actually may look like in Go source code.

For this tutorial (and the following parts), I am hosting a github repository here https://github.com/hariesef/myschool

It also serves as reference, examples and anybody later on can clone it, use it as bootstrap or scaffold to quickly write your business logic without having to worry about overhead in controller or repository setup too much.

Back to main topic. In Go best practices, basically only three folders are defined:

Folder: cmd

This is where the main package resides. The one we have to compile and build into binary. If we are going to delivery multiple binaries, i.e. one for REST API server, one for client test, the other one for CLI for instance, then the folder structure may look like this:

In this tutorial we will only implement the cmd/server/main.go.

Folder: internal

This is where we put our internal packages. Simply check these conditions:

• We do not want to share the packages to other projects
• We can’t share because depends on other internal packages
• The sub package can’t be copy pasted and expect to run PnP in other project

Folder: pkg

OK. If we have folder named internal, for sure we have opposite folder called external, correct? Yes but unfortunately the convention for externally-available packages, is named as pkg in Go. Before we put something in pkg folder, let us consider these conditions:

• Can be shared to other projects
• Have no dependency to internal packages
• Can be copy pasted and expect to run PnP

Sub Folder Proposal

Let us describe one-by-one what could be under the folders:

• model: This contains interface files which represent how we store the data into our storage. Normally every sub folder may have 1–1 relationship to a table in database. In more complex requirements, separate interface (later on to be implemented as repository) can be added that bridges multiple tables, in order to do more complex query like table joins.
• mocks: This is where the mock files get generated from the interfaces. In the tutorial I will be using gomock. (Alternatively out there there is also Mocker and Testify). Most of the interfaces are usually needed as part of business logic/ Services specific implementation, thus, internal. But if you are building importable-library and also providing the mocks to the user, this folder can also go to pkg.
• controller: This is where we put interfaces and implementations of like RPC, REST API, CLI, etc.
• repositories: This folder is only meant to contain setup and struct wrapper for storages. We don’t expect complex files or sub folders inside.
• storage: In sub folder we can define different type of storage. In this tutorial we will be using sqlite and mongodb.
• services: This is where we will invent new service that will contain all of our business logic specific use cases. Normally Services will interact with both Controller and Repository.
• helper: lastly, any independent libraries that can be widely and simply used by other packages, can reside here. Some example: encryption function, or functions to get and parse configurations out of environment variables.

That’s all for part 1, please proceed to next part of the tutorial: https://hariesef.medium.com/go-e2e-tutorial-part-2-models-and-implementation-via-gorm-19ac6f9104e6

Thank you for reading. Cheers 🍻