Go E2E Tutorial Part 5: Unit Test with SQL Mock and GoMock

In this part we will be learning on how to test GORM implementation, but using SQL Mock instead of real database. We will also learn how to test layer separation using GoMock. The files are hosted here for your reference: https://github.com/hariesef/myschool

Link to previous chapter: https://hariesef.medium.com/go-e2e-tutorial-part-4-twirp-rpc-c7bd8eeae925

In previous chapter we learned about Twirp methods implementation for Student Repo. For learning purpose, we will be testing the methods, with unit tests that utilize SQL Mock.

Twirp Server Implementation is basically a Proxy to real service. During real project I would not recommend to actually build unit test for them. There should be no business logics inside them, and they only need to forward the request to service or repo, therefore unit test here has no actual benefit against code quality. The reason we are doing unit test here now, is for sql mock tutorial only. Which, ideally, be put under service chapter. But in this series of tutorial later on, our Service unfortunately will connect to NoSQL DB/ Mongo.

var _ = Describe("Student Repo Implementation", func() {

 var mock sqlmock.Sqlmock
 var server *internalRPC.StudentRPCServer

 BeforeEach(func() {

  var mockDb *sql.DB
  mockDb, mock, _ = sqlmock.New()
  dialector := postgres.New(postgres.Config{
   Conn:       mockDb,
   DriverName: "postgres",
  })
  db, err := gorm.Open(dialector, &gorm.Config{})
  Expect(err).ShouldNot(HaveOccurred())

  studentRepoImpl := sqLiteStudent.NewRepo(db)
  repo := &repositories.Repositories{
   StudentRepo: studentRepoImpl,
  }

  server = &internalRPC.StudentRPCServer{Repo: repo}
 })

Please check full file content here: https://github.com/hariesef/myschool/blob/master/internal/controller/rpc/student/student_rpc_impl_test.go

Explanation for snippet above:

• mock and server are made globally, because they will be used by every test case
• If you still remember, our student repo implementation is using GORM, means we are free to use any SQL driver. In this example, we are using postgres driver instead.
• However, the driver connects to mockDb instead.

The actual test case will look like this:

   It("tests creating a user", func() {

    mock.ExpectBegin()

    query := regexp.QuoteMeta("INSERT INTO \"students\" (\"created_at\",\"updated_at\",\"deleted_at\",\"name\",\"gender\")")
    rows := sqlmock.
     NewRows([]string{"uid"}).
     AddRow(777)

    mock.ExpectQuery(query).WillReturnRows(rows)
    mock.ExpectCommit()

    student, err := server.Create(context.TODO(), &pkgRPC.StudentParam{Name: "Mama Ishar", Gender: "F"})
    Expect(err).ShouldNot(HaveOccurred())
    Expect(student.GetId()).To(Equal(int32(777)))
    Expect(student.GetName()).To(Equal("Mama Ishar"))
    Expect(student.GetGender()).To(Equal("F"))
    Expect(student.GetCreatedAt()).Should(BeNumerically(">", 1600000000))
   })

Explanation of above unit test:

• Variable mock is where we set the behavior of database operation
• Our main operation is the server.Create(). When this runs, the question will be, what are the underlying DB operations executed?
• The answer to that: GORM will actually perform transaction begin, then running query to insert into students table, then closing it with commit operation.
• This is why before hand, we are telling mock: hey mock, be prepared, you will be expected to run a Begin(), then if there is request to insert row to students table, please return back a row containing uid. Then finally you are expected to run Commit()

Working with DB Mock is a trial and error. At first we would never know what are the actual operations that will run.

The actual step in building this test case is actually like this:

Start with running directly the method:

mock.ExpectQuery("insert query")    
student, err := server.Create(context.TODO(), &pkgRPC.StudentParam{Name: "Mama Ishar", Gender: "F"})

Then just run the ginkgo, we expected to see an error, but the error will inform us what is going on:

 call to database transaction Begin, was not expected, next expectation is: ExpectedQuery => expecting Query, QueryContext or QueryRow which:
        - matches sql: 'insert query'

We are expecting a query, dummy one, called “insert query” to run, but in actual, GORM is doing transaction Begin().

Alright, let’s modify our test case to accomodate begin:

    mock.ExpectBegin()
    mock.ExpectQuery("insert query")

    student, err := server.Create(context.TODO(), &pkgRPC.StudentParam{Name: "Mama Ishar", Gender: "F"})

Then we will have different error on ginkgo:

Query: could not match actual sql: "INSERT INTO "students" ("created_at","updated_at","deleted_at","name","gender") VALUES ($1,$2,$3,$4,$5) RETURNING "uid"" with expected regexp "insert query"; call to Rollback transaction, was not expected, next expectation is: ExpectedQuery => expecting Query, QueryContext or QueryRow which:
  - matches sql: 'insert query'

In this recent error, ginkgo actually tells us the real operation done is INSERT INTO “students” (“created_at”,”updated_at”,”deleted_at”,”name”,”gender”) VALUES ($1,$2,$3,$4,$5) RETURNING “uid””

So instead of dummy query “insert query” ginkgo expect us that long string as the argument of ExpectQuery().

Please note that, the string match there is actually Regex match. Therefore normal character escaping will not work. To accommodate this, better use helper from QuoteMeta instead.

    mock.ExpectBegin()

    query := regexp.QuoteMeta("INSERT INTO \"students\" (\"created_at\",\"updated_at\",\"deleted_at\",\"name\",\"gender\")")

    mock.ExpectQuery(query)

    student, err := server.Create(context.TODO(), &pkgRPC.StudentParam{Name: "Mama Ishar", Gender: "F"})

In here I didn’t put complete query, because sub string is enough. It will be up to you on how strict you are comparing the query.

Running ginkgo now will give us different error:

Query 'INSERT INTO "students" ("created_at","updated_at","deleted_at","name","gender") VALUES ($1,$2,$3,$4,$5) RETURNING "uid"' with args [{Name: Ordinal:1 Value:1690688985} {Name: Ordinal:2 Value:1690688985} {Name: Ordinal:3 Value:0} {Name: Ordinal:4 Value:Mama Ishar} {Name: Ordinal:5 Value:F}], must return a database/sql/driver.Rows, but it was not set for expectation *sqlmock.ExpectedQuery as ExpectedQuery

We would focus on the error text “must return a database/sql/driver.Rows” — so it actually expects the query to return a row as result. So let’s modify again the test:

    mock.ExpectBegin()

    query := regexp.QuoteMeta("INSERT INTO \"students\" (\"created_at\",\"updated_at\",\"deleted_at\",\"name\",\"gender\")")
    rows := sqlmock.
     NewRows([]string{"uid"}).
     AddRow(777)

    mock.ExpectQuery(query).WillReturnRows(rows)

    student, err := server.Create(context.TODO(), &pkgRPC.StudentParam{Name: "Mama Ishar", Gender: "F"})

Executing this will give us our final error:

all expectations were already fulfilled, call to Commit transaction was not expected

It actually expects a Commit after the query. Adding the Commit() will finally make the testcase just like the original snippet on 2nd-top of this article.

Testing Interface Implementation using GoMock

Previous sub chapter talks about testing with real DB driver, it is just we alter the DB object, to give us custom result.

If we go one layer above, we are actually able to test any interface implementation using GoMock. Consider the diagram below:

Between Service and Repo, is separated by an interface. On right side, in storage implementation we would like to go full blown with online local DB in our unit tests, plus few DB Mock to emulate edge cases.

On left side, we are not suppose to instantiate the actual storage implementation, when are testing business logics inside the Service. We should mock the storage implementation and their return values.

Let’s go back to our original student interface file that can be found here: https://github.com/hariesef/myschool/blob/master/pkg/model/user_iface.go

On top of file we would find this line:

//go:generate mockgen -source=user_iface.go -package=mocks -destination=../../internal/mocks/user_iface.go

It is actually a tag that is used under Makefile, to generate mock files based on interface. The generated file will be under internal/mocks folder. The generated files are checked-in to github as well. But in case we want to generate them again, the above command can be run, or alternatively with make go-gen from the base directory.

The mock file can be used by unit test to emulate result of storage implementation.

Still talking about this test file, but now we are going on the next test group. We still want to test the server.Create() from Twirp implementation, but right now we will not be mocking the DB, instead the Repo.

 Describe("Testing student repo functions using Mock of repo", func() {
  Context("User Creation", func() {
   It("tests creating a user", func() {

    repoMock := mocks.NewMockStudentRepo(ctrl)
    repo2 := &repositories.Repositories{
     StudentRepo: repoMock,
    }
    server2 := &internalRPC.StudentRPCServer{Repo: repo2}

    repoMock.EXPECT().Create(gomock.Any(), gomock.Any()).Return(&sqLiteStudent.Student{
     Name:   "Haries",
     Gender: "M"}, nil)

    student, err := server2.Create(context.TODO(), &pkgRPC.StudentParam{Name: "Mama Ishar", Gender: "F"})

Explanation:

• In previous unit test, the repo instance was built from real package that connects to Mocked DB.
• In this test, the repo is built from NewMockStudentRepo() function.

When the actual method is called: server2.Create(), we are expecting the Repo to be returning something. This is why in previous line we are doing repoMock.EXPECT().<This Function Gets Called> — which it calls Create() function from the Repo.

The function requires two arguments, but we don’t care in this case about the arguments, that’s why we put (gomock.Any(), gomock.Any()) as the arguments. In the future you could also customize the response based on the arguments, i.e.

• call to repoMock.EXPECT().Create(A, B).Return(Result-X) while
• if there is a call to repoMock.EXPECT().Create(A, C).Return(Result-Y) instead

The return value is quite straight forward, just what is described in the interface. In this case is StudentModel. However StudentModel is an interface that can’t be instantiated, so we have to return implementation struct instead: Student{} which comes from storage:

import (
...
sqLiteStudent "myschool/internal/storage/sqlite/student"

Another way to formulate return value

In case the response needs more complex handling before deciding result value, alternatively this way of mocking can be used:

    repoMock.EXPECT().Create(gomock.Any(), gomock.Any()).DoAndReturn(
    func(ctx context.Context, args model.StudentCreationParam) (model.StudentModel, error) {
      studentObject := sqLiteStudent.Student{Name: "Haries", Gender: "M"}
      return &studentObject, nil
    })

Above code enables us to redefine the function and we can do anything inside before finally returning the result.

That’s all for this chapter. Please continue reading to next part of tutorial: https://hariesef.medium.com/go-e2e-tutorial-part-6-service-creation-with-mongodb-d5422ac6c6ee

Thank you very much for reading! 🍻 Cheers!