Go E2E Tutorial Part 2: Models and Implementation via GORM
In this part we will be discussing about how to create a model interface and implement it with GORM SQL DB Framework. The files are hosted here for your reference: https://github.com/hariesef/myschool
Link to previous part of tutorial: https://hariesef.medium.com/go-e2e-tutorial-part-1-clean-architecture-and-folder-structure-4ae6c486867c
Data Model
Let us start with a very simple example. If I want to have database table called Student, and it only needs to store two main information, that is, student name, and student gender, how would the model interface looks like in Go?
type StudentModel interface {
GetName() string
GetGender() string
}A model interface is only required to have functions to retrieve the basic data, like the name and gender. But in real implementation we would need the primary key or ID, right?
Sure, let’s add one:
type StudentModel interface {
GetUID() uint
GetName() string
GetGender() string
}Great, next is to define what are the basic operations we require on this table. It would be CRUD of course:
type StudentRepo interface {
Create(ctx context.Context, args StudentCreationParam) (StudentModel, error)
Read(ctx context.Context, uid uint) (StudentModel, error)
Delete(ctx context.Context, uid uint) (StudentModel, error)
FindByName(ctx context.Context, studentName string) ([]StudentModel, error)
}PS: I intentionally didn’t put example of “Update” since it is pretty straight forward. Instead what is more important is “FindByName” example above that may return multiple results.
OK. Now after having the StudentRepo it becomes clear what are the expectation that can be used by the Service. But why is the argument for creating the student, is StudentCreationParam? Shouldn’t string name, and string gender are enough?
Yes they are enough. But best practice for creation of an object must not hardcode the fields into arguments. Imagine if in the future we have to extend the content of Student, to add birthday, home address, classroom, etc. This will lead into major changes of interface, and also changes to interface implementations, and changes to the caller/ Services. Wrapping them into a struct is the solution, and it will look like this:
type StudentCreationParam struct {
Name string
Gender string
}Great. So where should we place this model code?
We will put it under pkg/model/ folder. Actual file can be seen here https://github.com/hariesef/myschool/blob/master/pkg/model/student_iface.go
How about the implementation files? The implementation can vary depends on the storage that will serve the data. In this tutorial for student model we will be using SQLite. Therefore the folder is:
PS: teacher and classes model are only figure of examples, to show the audience that if would like to have actual “MySchool” backend, it will need more than Student model.
Great, can now we see how to implement Student?
Hold your horses 😄 I would like to introduce you first to GORM, if you haven’t heard yet.
About GORM
GORM (link to official documentation) is Object Relational Mapping library for Go. It is super friendly and increases developers productivity because with GORM we don’t have to deal with lower layer queries preparations and parsings. It is available for MySQL, PostgreSQL, SQLite, SQL Server, TiDB and Clickhouse DB. Other SQL database which uses same SQL flavor like MySQL and PostgreSQL can use MySQL GORM Library and most likely it will also work out of the box (need PoC). And the best thing I like from GORM is: We don’t have to change our code if we want to switch DB from i.e. MySQL to Postgresql !
Amazing, right? GORM can also manage
- Unique Identifier/ Incremental/ Primary Key
- Indexes
- CreatedAt, UpdatedAt, DeletedAt fields automatically
- Soft deletion: deleting a record will not permanently remove the record from the table, but will set the DeletedAt field. The record with soft-delete won’t show in normal queries.
Now let us go back to our model file, to add GORM related fields. We can simply do this:
type StudentModel interface {
GetUID() uint //gorm internal feature
GetCreatedAt() int //gorm internal feature
GetUpdatedAt() int //gorm internal feature
GetDeletedAt() int //gorm internal feature
GetName() string
GetGender() string
}PS: yes, the UID will be also managed by GORM.
Model Implementation
Kindly hear my r̶a̶n̶t̶ proposal on how the file naming should be:
About Naming Convention
In many go repositories I often got lost when trying to understand the file and folder structure. When I want to see the implementation, clicking and opening the file shows me the interface instead 😅 This is only my proposal, but using this convention like we often see in Java, actually makes me easy to browse through project folder and finding what I need:
- <model/service/control>_iface.go : for interfaces declaration
- <model/service/control>_impl.go : for the actual implementation
- <model/service/control>_impl_test.go : for the unit test suite
In accordance to this tutorial, we will be implementing two files:
- internal/storage/sqlite/student/student_repo_impl.go
- internal/storage/sqlite/student/student_repo_impl_test.go
The following is the snippet from student_repo_impl.go where we implement the StudentModel interface:
// implmentation model of Student
type Student struct {
UID uint `gorm:"primaryKey;autoIncrement"`
CreatedAt int `gorm:"autoCreateTime"` //store as second unix timestamp to have compatibility with SQLite
UpdatedAt int `gorm:"autoUpdateTime"`
DeletedAt soft_delete.DeletedAt `gorm:"index"`
Name string
Gender string
}The tag gorm specifies that the field is managed by GORM. On UID, we specify this is primary key and it is auto incrementing. Normally we would put at least uint64 as the type. But to make it compatible with SQLite in this tutorial, we are using standard unsigned int.
Updated and DeletedAt will be automatically updated by GORM due to the tags we are using on them. While DeletedAt, additionally will create index, and it actually needs to be indexed if we want to include soft-delete functionality, because table search query will always scan this field.
PS: to make it easier for developers, declaration of UID, CreatedAt, UpdatedAt, DeletedAt can be removed, and replaced with single line gorm.Model. I intentionally put them here for learning purpose. More information about gorm.Model can be seen at GORM documentation.
Declaring the struct is not enough, because we have to also declare the basic getter, like in our model interface:
func (s *Student) GetUID() uint {
return s.UID
}
func (s *Student) GetCreatedAt() int {
return s.CreatedAt
}
................Getting hint from my colleague fudanchii (thanks bro! 🍺) the following is one liner to ensure that our file has implements everything required from the interface:
// Safe checker to know if this file already implements the model interface correctly or not
var _ (model.StudentModel) = (*Student)(nil)If we missed a method, i.e. GetCreatedAt() forgot to be added in implementation, the above line will give error. It basically tries to assign a nil variable with type of Student, into unused/nil placeholder with type of the interface.
Table Creation
There are two ways to populate the initial database. The first, is to ask GORM to create the table for us. As example, and I am using this for unit testing, can be seen here:
BeforeSuite(func() {
db, err = gorm.Open(sqliteDriver.Open("localtest.db"), &gorm.Config{})
if err != nil {
errString := "Cannot access localtest.db: " + err.Error()
panic(errString)
}
studentRepoImpl = sqLiteStudent.NewRepo(db)
db.Migrator().DropTable(&sqLiteStudent.Student{})
db.Migrator().CreateTable(&sqLiteStudent.Student{})
ctrl = gomock.NewController(t)
})For production site, I would not recommend using inline code to setup and automate the database migration, even if there are articles out there supporting this concept to make consistency between models in code, and actual tables in database. Database migration is a complex topic and needs to be handled carefully. Source code, in other hand is always expected to fail (zero trust). That’s why we have so many stages of automation and tests before deploying to production. The failure of DB migration caused by bug in code will cause catastrophic effect because there are so many steps at once during automation.
In larger organization where exists sole role of Database Engineer or Database Administrator, they are the one to review the database migration SQL scripts (and not Go source code) and execute them carefully. In more advanced pipeline where database migration is automated and be part of CI/CD, then we would need
- versioning of migration script
- schema.v02.sql, delta.v02.sql, and rollback.v02.sql
There is also certain tool like goose that organizes those into single SQL version.
A proper migration tool shall be able to upgrade from v01 to v05, and vice versa, to do rollback from 05 to 01. But this is another topic for another time 🏃
Back to topic, in this example SQL file is provided for schema creation. The file exists here https://github.com/hariesef/myschool/blob/master/pkg/storage/sqlite/schema/create.sql
I would recommend you to do some experiment with Go model struct, and what happens when GORM creates them in DB using db.Migrator().CreateTable(). The schema created by GORM might be just exactly what you need, … or not. It is fact that using SQL script we are more flexible in defining the schema and indexes.
Implementation with GORM
Previously we have seen implementation of the StudentModel, but we still need to implement the StudentRepo.
// unexportable
type repoPrivate struct {
db *gorm.DB
}
// Safe checker to know if this file already implements the interface correctly or not
var _ (model.StudentRepo) = (*repoPrivate)(nil)
// public constructor, requires actual database object
func NewRepo(db *gorm.DB) model.StudentRepo {
return &repoPrivate{db: db}
}The constructor (NewRepo) will return repoPrivate (unexportable) that actually implements model.StudentRepo. This means, the repoPrivate has to implement all needed functions like Create, Delete, etc.
The db object, instance of *gorm.DB is the only object we need to perform all database operations here. Let us see how to implement the Create() function:
func (repo *repoPrivate) Create(ctx context.Context, args model.StudentCreationParam) (model.StudentModel, error) {
studentObject := Student{Name: args.Name, Gender: args.Gender}
result := repo.db.Create(&studentObject)
return &studentObject, result.Error
} With GORM is it amazingly simple. To actually add new row in a table, we only need to call the db.Create() 😘
You can check rest of implemented functions here: https://github.com/hariesef/myschool/blob/master/internal/storage/sqlite/student/student_repo_impl.go
One thing I would like to emphasize is the implementation of FindByName. This function actually only serves as tutorial on how to query and return multiple rows. For example, find all students with name “eve”, it may return “Steve”, “Evelyn”, “Reves”, etc.
The tricky part is on the pointer handling.
func (repo *repoPrivate) FindByName(ctx context.Context, name string) ([]model.StudentModel, error) {
var foundStudents []*Student
result := repo.db.Where("name LIKE ?", "%"+name+"%").Find(&foundStudents)
models := make([]model.StudentModel, len(foundStudents))
for i, v := range foundStudents {
models[i] = model.StudentModel(v)
}
return models, result.Error
}Note that: we declare var foundStudents with type of []*Student and not with type of []Student. Although the one expected as return is array of model.StudentModel.
If we scroll back above, and check how we implemented Student functions:
func (s *Student) GetUID() uint {
return s.UID
}
func (s *Student) GetCreatedAt() int {
return s.CreatedAt
}
................The functions are implemented using pointer (s *Student). This is so we are sharing same object of struct, and not by copy of object.
This means, the equivalent implementation of
model.StudentModel is actually pointer of Student (*Student)
That’s why when we are assigning this:
models[i] = model.StudentModel(v)— it is accepted by Go. “v” type is actually *Student. It goes fine when it is being casted to type model.StudentModel, and stored in the result array models[].
Similar situation in the Create function actually:
func (repo *repoPrivate) Create(ctx context.Context, args model.StudentCreationParam) (model.StudentModel, error) {
studentObject := Student{Name: args.Name, Gender: args.Gender}
result := repo.db.Create(&studentObject)
return &studentObject, result.Error
}We are creating studentObject with type of Student{} — but then when returning it to the function, we do &studentObject so that it returns the pointer address instead. And it is equivalent with expected return type of the interface: model.StudentModel.
That’s all for this chapter. Please continue to the next part: https://hariesef.medium.com/go-e2e-tutorial-part-3-unit-testing-with-ginkgo-and-gomega-for-gorm-implementation-sqlite-473b5bb0a625
Thank you for reading 🍻 Cheers!
