Go E2E Tutorial Part 6: Service Creation with MongoDB

In this part we will be learning on how to create Service layer, that will utilize repositories connected to MongoDB. The files are hosted here for your reference: https://github.com/hariesef/myschool

Link to previous chapter: https://hariesef.medium.com/go-e2e-tutorial-part-5-unit-test-with-sql-mock-and-gomock-c53f4bf2d72f

For tutorial purpose we will be creating a service called Account service. If we go back to our first clean diagram:

We are here now. Look at the circled one.

Our imaginary Account service will be serving three services: Create User, Login (authenticate), and to Logout.

This will be the interface for our service:

package account

import "context"

type AccountServiceIface interface {
 Create(context.Context, string, string) error
 Login(context.Context, string, string) (*TokenInfo, error)
 Logout(context.Context, string) error
}

type TokenInfo struct {
 Token  string
 Expiry int
}

File location: https://github.com/hariesef/myschool/blob/master/pkg/services/account/account_iface.go

• Create will require email and password
• Login will also require email and password, and returning Token information
• Logout will require token to remove

Now in order to do user creation, and then login logout functions, at least we will require two models/ repositories:

• User, to store credential information
• Token, to store token when a user has successfully logged-in.

This will be our implementation, please refer to the github repo for complete file content:

User repo:

• pkg/model/user_iface.go
• internal/storage/mongodb/user/user_repo_impl.go
• internal/storage/mongodb/user/user_repo_impl_test.go

Token repo:

• pkg/model/token_iface.go
• internal/storage/mongodb/token/token_repo_impl.go
• internal/storage/mongodb/token/token_repo_impl_test.go

Please notice that, Repo task is to only do CRUD of the provided data. Nothing less, nothing more. If asked to Create(“haries”, “abcdefg”), then the Repo should store that into DB as is, with password “abcdefg”. The business logic including: password hashing, token expiry date handling, etc must be done in Service layer, not in Repo.

MongoDB Implementation

We will be picking one repo implementation: user_repo_impl.go to learn how to do CRUD with Mongo.

File location: https://github.com/hariesef/myschool/blob/master/internal/storage/mongodb/user/user_repo_impl.go

For mongo, the implementation of User model needs certain tags:

// implmentation of UserModel
type User struct {
 ID                string `json:"id,omitempty" bson:"_id,omitempty"`
 Email             string `json:"email,omitempty" bson:"email,omitempty"`
 EncryptedPassword string `json:"encryptedPassword,omitempty" bson:"encryptedPassword,omitempty"`
 Active            bool   `json:"active,omitempty" bson:"active,omitempty"`
}

• Mongo works with BSON instead of JSON. We need the bson tag to specify the actual field name in DB collection.
• _id is the default “unique primary key” of a collection that will be auto-generated
• omitempty means if we are creating or updating a document (row) inside collection (table), if the field is not present or null, it will be skipped, instead of keep setting the field as null.

// unexportable
type repoPrivate struct {
 db mongo.Database
}

// Safe checker to know if this file already implements the interface correctly or not
var _ (model.UserRepo) = (*repoPrivate)(nil)

func NewRepo(db *mongo.Database) model.UserRepo {
 return &repoPrivate{db: *db}
}

func (repo *repoPrivate) Create(ctx context.Context, args model.UserCreationParam) (model.UserModel, error) {

 newUser := &User{
  Email:             args.Email,
  EncryptedPassword: args.EncryptedPassword,
  Active:            true,
 }
 res, err := repo.db.Collection(UsersCollectionName).InsertOne(ctx, newUser)
 if err != nil {
  return nil, err
 }
 newUser.ID = res.InsertedID.(primitive.ObjectID).Hex()
 return newUser, nil
}

Creation of new document in Mongo is very straight forward, just use InsertOne(). Alternatively there is also batch insert with InsertMany(). More on this can be read on https://www.mongodb.com/docs/drivers/go/current/fundamentals/crud/write-operations/insert/

Creating new document will always return its primary key/ ID.

Please note that, the primary key type in Mongo is ObjectID, and not a string. That’s why to read it, we will need conversion.

To read a document based on email as search key, we use the following code:

func (repo *repoPrivate) Read(ctx context.Context, email string) (model.UserModel, error) {
 
 result := repo.db.Collection(UsersCollectionName).FindOne(context.Background(), bson.M{"email": email, "active": true})
 var user User
 result.Decode(&user)
 return &user, result.Err()
}

Next function, Deactivate() is where we are trying to find a document based on primary key “_id”. This function will set user’s Active field, to false.

func (repo *repoPrivate) Deactivate(ctx context.Context, id string) (model.UserModel, error) {
 objectId, err := primitive.ObjectIDFromHex(id)
 if err != nil {
  return nil, err
 }

 filter := bson.M{"_id": objectId}
 update := bson.D{{Key: "$set", Value: bson.M{"active": false}}}
 opts := options.FindOneAndUpdate().SetReturnDocument(options.After)
 result := repo.db.Collection(UsersCollectionName).FindOneAndUpdate(context.Background(), filter, update, opts)
 var user User
 result.Decode(&user)
 return &user, result.Err()
}

BSON has several primitive types, like bson.M and bson.D.

• bson.M has format of map
• while bson.D is actually array of key-value. In Go the literal usage of Key: and Value: is required to remove certain linting warning.
• More on primitive types can be seen here https://www.mongodb.com/docs/drivers/go/current/fundamentals/bson/

The final function for this tutorial will be to find Users that are still active:

func (repo *repoPrivate) FindActive(ctx context.Context) ([]model.UserModel, error) {
 var users []*User

 filter := User{Active: true}
 opts := options.Find().SetSort(bson.D{{Key: "email", Value: 1}})

 cur, err := repo.db.Collection(UsersCollectionName).Find(ctx, filter, opts)
 if err != nil {
  return nil, err
 }
 if err := cur.All(ctx, &users); err != nil {
  return nil, err
 }

 models := make([]model.UserModel, len(users))
 for i, v := range users {
  models[i] = model.UserModel(v)
 }
 return models, err
}

Unit Testing with Online Mongo DB

Setting up the suite will look like this:

 BeforeSuite(func() {

  var err error
  opt := options.Client().ApplyURI("mongodb://localhost:27017")
  localMongoClient, err = mongo.Connect(context.Background(), opt)
  Expect(err).To((BeNil()))

  err = localMongoClient.Ping(context.Background(), readpref.Primary())
  Expect(err).To((BeNil()))

  db := localMongoClient.Database("unit_test_db")

  //initialize DB
  db.Drop(context.Background())

  //make email as unique index
  indexModel := mongo.IndexModel{
   Keys:    bson.D{{Key: "email", Value: 1}},
   Options: options.Index().SetUnique(true)}
  _, err = db.Collection(user.UsersCollectionName).Indexes().CreateOne(context.TODO(), indexModel)
  Expect(err).To((BeNil()))

  userRepoImpl = user.NewRepo(db)
  ctrl = gomock.NewController(t)
 })

Full file content can be seen here: https://github.com/hariesef/myschool/blob/master/internal/storage/mongodb/user/user_repo_impl_test.go

Explanation:

• Need a mongo or docker running on localhost with default port 27017
• after the client connected, do a quick ping test to see it works
• then set the database name to do the test
• drop the DB to clear all content
• setup indexing based on email field.
• Initialize the UserRepo object ← our testing target

Testing online DB is also straight forward:

  Context("User Repo", func() {

   firstUserId := ""
   randomPassword := helper.RandomStringBytes(5)
   It("tests creating first user", func() {

    user, err := userRepoImpl.Create(context.TODO(),
     model.UserCreationParam{Email: "haries@banget.net", EncryptedPassword: randomPassword})
    firstUserId = user.GetID()
    fmt.Println("first user id: ", firstUserId)
    Expect(err).To((BeNil()))
    Expect(len(firstUserId)).Should(BeNumerically("==", 24))
    Expect(user.IsActive()).To(Equal(true))
   })

Rest of test cases are:

2) test creating another user with same email, failed because email has to be unique

3) test reading back our first user, matching the random password and userID we had before

4) test deactivating our first user

5) test reading back our first user ← failed no document found

6) test the last function to find only active users

Account Service Structure

Now we have seen how the User Repo was built and tested, let’s move the actual service.

PS: Token Repo is subject to test using different method in next part of tutorial.

Following what we have done in this tutorial so far, creating account service has very similar steps: defining the interface, implementing the service, unit testing, defining the proto file (because we want to serve this), and implementing the twirp methods, to proxify call to actual service.

Here is snippet on how the user creation in Account service:

func (acct *AccountService) Create(ctx context.Context, email string, password string) error {
 token := ctx.Value("token")
 log.Debugf("value of token: %s", token)
 hashedPassword := argonFromPassword(password)

 _, err := acct.Repo.UserRepo.Create(context.TODO(),
  model.UserCreationParam{Email: email, EncryptedPassword: hashedPassword})

 return err
}

• token is example on how the we are getting the request header value. Previously we discussed about the header is being stored by Twirp optional handler. This line only serves as example and nothing really relates with actual business logic.
• we use custom function argonFromPassword() to hash, that is part of implementation file.

Full file content can be seen here: https://github.com/hariesef/myschool/blob/master/internal/services/account/account_service_impl.go

func (acct *AccountService) Login(ctx context.Context, email string, password string) (*account.TokenInfo, error) {

 //first read the user data
 user, err := acct.Repo.UserRepo.Read(context.TODO(), email)
 if err != nil && errors.Is(err, mongo.ErrNoDocuments) {
  return &account.TokenInfo{}, errors.New("email or password does not match our record")
 }
 if err != nil {
  return &account.TokenInfo{}, err
 }

 log.Debugf("[%s]", user.GetEmail())
 log.Debugf("[%s]", email)
 log.Debugf("[%s]", user.GetEncryptedPassword())
 log.Debugf("[%s]", argonFromPassword(password))
 //then, compare
 if (user.GetEmail() != email) || (user.GetEncryptedPassword() != argonFromPassword(password)) {
  return &account.TokenInfo{}, errors.New("email or password does not match our record")
 }

 //login successful, now generate a random token
 newToken := helper.RandomStringBytes(32)
 newTokenExpiry := time.Now().Unix() + (3600 * 24) //one day
 //store the token

 _, err = acct.Repo.AuthTokenRepo.Create(context.TODO(), model.TokenCreationParam{
  Token:  newToken,
  UserID: user.GetID(),
  Email:  email,
  Expiry: int(newTokenExpiry),
 })

 if err != nil {
  return &account.TokenInfo{}, err
 }

 return &account.TokenInfo{Token: newToken, Expiry: int(newTokenExpiry)}, nil
}

func (acct *AccountService) Logout(ctx context.Context, token string) error {

 return acct.Repo.AuthTokenRepo.Delete(ctx, token)
}

Login function is little bit complex:

• check if user exists or not
• compare email and hashed password
• setup token
• return result

While Logout function is really straight forward, one liner.

Unit Test for account service is still using online mongo DB connection to localhost, and will not be elaborated again. If you are interested, please have a look at here: https://github.com/hariesef/myschool/blob/master/internal/services/account/account_service_impl_test.go

Serving Account Service with Twirp

Serving the account with Twirp is same like we were serving the Student Repo:

 accountService := accountSvcImpl.AccountService{Repo: repo}
 accountRPCServer := &accountRPCImpl.AccountRPCServer{AccountSvc: &accountService}
 accountTwirpHandler := accountRPCIface.NewAccountServer(accountRPCServer,
  rpc.TwirpHookOption(accountRPCIface.AccountPathPrefix))
 accountTwirpHandler2 := rpc.WithEvaluateHeaders(accountTwirpHandler)

 router := chi.NewRouter()
 router.Mount(accountRPCIface.AccountPathPrefix, accountTwirpHandler2)
 router.Mount(studentRPCIface.StudentPathPrefix, studentTwirpHandler2)

• create account service object which depends on repositories
• create RPC server, with account service as input
• create twirp handler with optional hook function
• wrap twirp handler with headers handler
• mount the twirp handler on different path

Then it can be tested using Postman:

That’s all for this chapter. Next we will be unit-testing Token Repo using different method, MongoDB Mocking: https://hariesef.medium.com/go-e2e-tutorial-part-7-unit-testing-with-mongodb-mock-mtest-e32511961925

Thank you reading! 🍻 Cheers!