Let’s keep the distances

In my last article I mentioned that we had cases when we needed to split our back-end application across several processes and as a result we gained more flexibility when we needed to scale the application. This time I want to keep sharing some insights on why we try to keep some parts of the application loosely coupled. This time I wanted to talk about loose coupling at design time when the whole application might run in the single process. According to one of the definitions ”loosely coupled system is one in which each of its components has, or makes use of, little or no knowledge of the definitions of other separate components”. In object oriented paradigm, that we are using to create our Fitradar application, most of the code is spread among various classes and therefore the smallest component that we consider is a class. And since classes are just containers for data and functions then we usually see the loose coupling as a way how to call a function of another class with little information as possible.

The simplest way how to call a method on the other object is to create that object from a class (in Java and C# we do that by new operator) and call the needed method. But it turns out there are many cases when it is undesirable to know the class name of an object or even a specific method name during the design time and therefore there are several methods how to reduce the knowledge of a class we are going to use and a method we are going to call:

  • We can replace a class with an interface or an abstract class and in such way reduce the information about the class to the subset of methods that the other classes are interested in. In Object Oriented programming this is known as Polymorphism and is one of the OOP cornerstones. It is really hard to express how important this OOP feature is. I think back in a day that feature was one of the selling points of OOP that pushed out the procedural programming. The biggest gain of using Polymorphism is a flexibility. Now we are not bound to a single code execution flow, because every function that is presented in the execution flow via interface or abstract class can be replaced with other implementation through the configuration or dynamically during a runtime. This in turn opens whole bunch of new possibilities. And the one I like the most is Unit testing. Now days it is hard to imagine how much work it would require to isolate the piece of code and test it without the interfaces that can be mocked.
  • Sometimes we are interested only in one method of the other class and we don’t want to be bound to single class that implements this method, and we want to replace that method implementation. In order to reduce the knowledge about the other class to single methods signature we can use delegates in C#, functional interfaces in Java, Kotlin or function pointers in C\C++. Not all OOP languages provide such constructs though then we should use more verbose interfaces or abstract classes. A delegate can bee seen as placeholder for a method. And once we start to focus on a method without accompanying class we start to enter the realm of Functional programming where function implementation can be written as lambda expression (anonymous functions) and its return type and arguments can be other functions. The most common areas where the function placeholders are used are events and callbacks. And once Linq came out everyone who got to know this library really started to appreciate lambda expressions
  • In cases where Interfaces and Anonymous methods are not enough to describe dependency, for example we don’t want to restrict a method description in our execution flow with a single signature and want to use different rules to describe the method or a class that contains the method, then we can use Reflection. Although Reflection is very powerful tool that allows to achieve a great flexibility, usually it is not recommended to use because it adds an overhead at runtime and degrades the performance

On the larger scale where components are not anymore single classes but the whole systems the loose coupling further can be achieved by introducing the middle-ware like Queues and Service Buses.

As we can see making a method call something that is not bound to a single implementation opens up a dozen of options how to design and build an application in very flexible way.

P.S. Visit http://fitradar.me/ and join the mailing list!

Stay Loose

Recently our team finished the work on FitRadar booking system. That was quite a challenging task and I decided to share some insights and knowledge we gained during the design and development of this system. Already on the early stage of designing and requirement gathering it was clear that ticket booking system on the back-end should be independent from the rest of the Fitradar system because we wanted to introduce Event Sourcing for such entities as Order and Payment. Since these two entities are responsible of money charging and money is always a very sensitive topic, we wanted to make sure we can always track down the state changes in booking and payment processes. But to what degree it should be independent was the question we had to answer.

Let’s start with a simple case we considered – one process application. In this kind of applications all communications between objects occur within one process and it means all libraries needed for application are loaded in single process. Usually when we talk about the client side applications – Android, iOS or client side JavaScript, then most of the time these kind of applications work within one process and rarely leave the boundaries of that process set by OS. In such case the isolation can be introduced on the code level, like implementing the new feature in a new library, but at runtime we are somewhat limited. Since all objects share the same process:

  • fault in one object might lead to crash in another object
  • the load of a single object affects the whole process, so we can’t assign PC resources to separate app components. It means you can only scale the whole application, usually by creating a new instance of the application. This starts to play a big role when app is hosted on cloud environment like Azure, Google Cloud or AWS where we must pay for used resources.
  • It is hard to apply new technology or framework to the new feature. Usually the whole application is build based on single architectural pattern like MVC and CRUD or MVC and CQRS and it is hard to introduce one more pattern (if we already using simple Data Access Objects for working with data coming from web, then it will take some time to introduce CQRS with Event Sourcing) in the same application. It is much simpler to create a new application.

So there is very little we can do about object decoupling at runtime in single process application case, but do we really need to deploy a booking system in a separate application? Maybe we can just isolate our booking system in a separate library and still run it in the same process with the rest of FitRadar libraries? And so we started to investigate it. Very quickly we noticed that our ideas of making booking system independent from the rest of application strongly reassembles the principles of Microservices architecture . And since now days there are a lot of talks around the Microservices we thought that our project might be a good candidate to move to Microservices architecture. But just because something is promoted doesn’t mean it fits your needs. You usually don’t start the development of application as a distributed system, unless it was a requirement from the beginning. Applications usually mature to the state when Monolithic Architecture doesn’t satisfy the initial requirements and a team starts to consider switching to Microservices Architecture. And we wanted to make sure that this is the time when the new feature requires us to switch to the new Architecture to make the development easier in the future. Here are the advantages of Microservices Architecture that really attracted our attention:

  • it gives us a smaller code base to work with. It is a really huge benefit if you consider that at certain point a team more and more starts to think about how properly structure the project (how to write the Clean Code), that any new developer could easy understand it and navigate it. And putting a Booking system as a separate solution would allow as to reduce the code complexity and as result would allow developers navigate faster and add new code faster. If you have ever worked on enterprise scale application development then you know how much time it can take just to figure out where to put your new code.
  • Web API end to end tests and Integration tests can run much faster because there is no need to load all of the Fitradar system libraries only those related to booking system. This advantage really starts to shine when we enter the test phase.
  • It allows us to scale booking system and the rest of the back-end application independently. The booking system is more focused on data writing in the database on the other hand main back-end application is focused on data serving. We expect thousands of data requests a day while user will be navigating around in mobile application and just few bookings a day.
  • Improved fault isolation

And for our project it seemed that we will gain more than loose if we will treat booking system as separate application and deploy it in a separate process. So we gave it a try.

Visit our website: http://fitradar.me/ and join the mailing list. Our app is coming soon!

Mobile app development versus back-end app development

One day I was listening interview with a person who was running software development courses, he was answering questions regarding the software development in different fields and what a full stack developer means. And since our Fitradar system covers several development fields I decided to share my experience about working in one or another field and how easy is to jump from one domain field to another one.

I remember back in a day when I was still a student at university several classes were focusing on business modeling and system implementation according the model. As later I found out this was a common approach for enterprise application design and development. But my first job related to software development was in the company that produced routers and its software. At that time I was writing small scripts to test different router configuration setups, I was not directly involved in router’s software development but I wanted to become a par of a development team. And so I started to explore the routers operating system which was based on Linux kernel as many other embedded systems. I soon discovered that many principles I learned in programming classes are not applied in embedded systems and instead big emphasize is put on performance. For example instead of throwing exceptions error codes were used to improve the performance. In mobile and enterprise applications that would be considered as a bad practice. Another odd thing for me was to see that there are no unit tests only integration or end to end tests, which again in enterprise application world would be considered very bad. And so at that moment it hit me that the way how one or the another system is developed depends from the domain field and not that much on the language. So for myself I distinguished following software development fields:

  • web front-end development
  • mobile application development
  • desktop application development
  • game development
  • embedded systems development
  • enterprise application development

This is by no means the full list of the domain fields where the software is developed. These are just areas I have come across in my developers career. There are many principles that cover all the fields and that is what every programming class starts with, like variables, loops, conditions, functions, but once we need to organize bigger code base the principles start to vary. In our Fitradar project we are developing two big applications: mobile application for Android and iOS and back-end system that gradually evolves in full scale enterprise application. The approaches in some parts of both systems are similar but some parts have quite different goals. So in this article I wanted to sum up the differences and common things approaching the mobile app and back-end app development in Fitradar solution. As mentioned earlier those differences really start to show up when the code base starts to become so big that you need some extra time to navigate within it. And if you don’t follow any code organization principles the time you will need to navigate around, understand and modify will grow proportionally and sometime even exponentially to the size of the code base. So for big systems we really need to organize our code. And almost all my previous articles about development were dedicated to the different approaches on how to better organize the code. The one thing I discover again and again is this – although it is important to know the principles like Object Oriented programming, SOLID and design patterns but just as important is to apply those principles only there where is needed. I remember one web project where front-end part was developed in Angular but back-end in ASP.NET MVC. Our team took over the project from the other company and had to continue to extend it with new features. When we worked with back-end part it was easy to understand and modify it because it followed well established enterprise application best practices. But we really struggled with front-end part because the code was organized according the same principles as in a back-end part and it looked like developers were ignoring many Angular built-in features and principles. And only later we found out that the developers who mainly worked with back-end designed the front-end application as well. This approach would have worked fine if the front end had been very simple application but it was so big that in order to organize the front-end code it required a knowledge of principles specific only to User Interface. And from my experience for a developer new in a domain field to start to produce decent design takes about a year and more, not counting the time needed to master the programming language itself. So therefore for big web application projects there are usually front-end and back-end developers. For smaller web applications the same knowledge about the programming might be enough. So let’s see where the focus in back-end and mobile application lays in:

  • as you can imagine even the simplest mobile app has an UI (there are some special background apps but we will not consider them here) and therefore the emphasize in mobile application in first place will be on the UI code organization and how to connect UI with the rest of the application. The UI will be the part of the system that takes the input from a user and displays the information to the user. From the other hand back-end interaction with the outside world will be via REST (or maybe GraphQL) web services where data is received and send in well formatted way. And formatting usually is done by back-end third party library. Since UI can be very complex then we need to consider principles, practices and patterns that are specific only to UI development. And that is where pure back-end developers lack the knowledge. And if no data is used then mobile app might be limited just to UI and for back-end developer that would mean that very little knowledge can be transferred from back-end development. But in case mobile application works with data stored either locally on mobile device or on a back-end server the extra layer of data persistence might be required.
  • Data layer development in mobile app and on the back-end might seem very similar. And indeed if we choose to store data on a mobile device we can chose database for this purpose and use the well known patterns like Repository and Data Access Object as we do on the back-end server, then it might give the impression that mobile app developer can develop this part of the system for both mobile and back-end application. But my experience shows that it is true only to some extent. On the mobile app data persistence layer always should be simple, because the hardware resources are very limited and it is unwise to build large scale database on the mobile device, instead data are transferred to back-end server and stored there. Database on mobile devices often is used as a cache store, where data are denormalized and structured purely for UI needs. On the other hand the back-end database design is a big thing where data normalization and performance is considered. And this time pure front-end developers might lack the knowledge of complex persistence layer design.
  • And when the system’s complexity grows more layers on the back-end start to emerge, like Domain layer and Event Bus. Inner details of Domain logic usually is something that people don’t want to expose to the outside world therefore it is implemented only on a back-end server. Domain logic implementation might require a lot of specific design patterns and practices. Which again only back-end developers might be aware of.

So at the end of the day we can still apply general software development knowledge across the domains as long as the code base stays small and simple. And that is why even high school student can produce decent software in any field as long as that peace of software is small. But once the system grows big particular domain expertise starts to become crucial, and that domain expert knowledge comes only over the years as a result of learning and practice.

Please visit our website: http://fitradar.me/ and join the mailing list! Our app is coming soon.

Repositories and Data Access Objects are still alive

In my previous articles when I was talking about Clean architecture and Domain Driven Design I mentioned one piece of domain layer – repository or more precisely repository pattern. The repository pattern is used to persist and retrieve domain models. Although repositories are mostly related to Domain Driven Design the other architectures might have the objects with the same functionality but are called different. In Fitradar application (front end and back end) the objects that are used to persist data in the database and fetch data from the database and map data to in-memory objects (aka POCO – Plain Old C# Object, POJO – Plain Old Java Object) and are not part of the Domain layer are called DAO (Data Access Objects). DAO usually are used in cases when no business logic is involved and the application needs to execute simple CRUD operation.

In Android application to work with database we use Room Persistence library but in our back-end solution we use Entity Framework Core. These two are know as Object-relational mappers or ORM. And today I wanted to explore the relationship between Domain Repositories, Data Access Objects and ORM and share some experience we had in our team working with these patterns and libraries.

If you look at the responsibilities of Repository, DAO and ORM they are very close to each other. Some years ago when ORM technology was in its inception the DAO and Repositories usually were working directly with the low level database access services. In case of .NET it was ADO.NET library and in case of Android it was SQLite library. And it was quite clear that DAO or Repository should use these libraries to persist or retrieve in-memory objects and map the data. But now-days when in application development main data access technology became ORM the border between ORM and DAO and Repository has become very blur, especially when one just switched to ORM. Working with ORM in different projects and languages our team came to conclusion that in same cases ORM can fully replace the DAO or Repository and can be used instead of it. Let’s look closer at the cases when it is appropriate to use just a ORM library and when the DAO or Repository should be used in combination with ORM:

  • If you need to save, update or delete single flat plain old object then in most cases ORM library will do the job for you. Of course EF Core and Room are capable of doing more, but then we really should investigate each case separately
  • If you need to fetch the data from single table by primary key then again in most cases you can fully relay on ORM
  • In case Domain layer aggregate has complex entity cluster hierarchy, where different entities might have different persistence state you most likely will need to write such aggregate persistence logic by yourself either using ORM or low level database access library. In our application one such aggregate is Sport Event that has Place and Organizer and some other entities. And the problem we faced when tried to save Sport Event by using built in EF Core capabilities was the different Entity State and the calculated state before Save operation. For example we had cases when we needed to create a Sport Event in Place that was not yet saved in database, and since both entities have the same Entity State EF Core tackled that case well and was able to create both entity records in database. But the problem started to appear when we wanted to create Sport Event in the Place that already had other Sport Events, now before saving aggregate Sport Event we had to fetch Place entity to make sure EF Core sets the correct Entity State. And the more entities your aggregate root has the more fetch operations might be needed. So in this case for us seemed obvious putting Entity State synchronization logic in separate Repository.
  • In case data query logic and following mapping logic is so complex that to make a code readable it is necessary to split logic in separate functions, you most likely will move those functions to separate class, Data Access Object, to persist the Single Responsibility principle.

As you can see the Repository and DAO still has their place in the modern software architecture. In case of queries in CQSR architecture query logic might be put in Application layer since anyway contrary to Commands, Queries only responsibility is data fetching, and that duplicates DAO responsibility. By choosing to put query logic in Application layer we make a direct dependency on EF Core. That is not a problem in classical three layer application, but it does not fit the Clean Architecture principle where Application layer should be aware only of Domain layer and with outer layers (EF Core belongs to the Infrastructure layer) communicates only through interfaces. In this case we should use DAO.

Please visit our website and join the mailing list. Our app is coming soon:

http://fitradar.me/

Clean architecture for our back-end

Today I want to continue discussing the architecture we came up with for our back-end service. In the previous article I brought up some arguments why it was important for us to separate read and write models. And the main thing that made it clear was that we operated with two different data sets for read and write operations. But I didn’t go much in to the details on how we arrived at such a different models. And this aspect of CQRS will be the topic of today’s article. Before we even realized we need to apply CQRS pattern in our design we decided to follow Uncle Bob’s Clean Architecture the same architecture we used in our mobile application development.

And the the reasoning that brought us to this architecture I lied down in one of my previous articles. Although on both platforms the overall architecture was the same but implementation details were quite different. The main attraction point of this architecture was possibility to build the server side application around the Domain Model and apply Domain Driven Development. There are several definitions for Domain Model and some of them you can find here but for us in the design process more important was to understand the peaces that constitute the Domain model. It allowed us in the next designing steps to decide whether we need a separate Domain model layer or entities were just enough. Quite often I notice that the architecture does not make a big difference between the bare bones domain entities and full domain model. In first case entities are just a database relational model, and they serve as in memory tables. In many cases especially in ASP.NET world (the framework we are using to build our back-end services) Object Relational Mappers like Entity Framework require to separate entities from the CRUD operations and thus giving impression that full-fledged Domain model is created, but in fact you end up with something what is called Anemic Domain model which is considered an anti-pattern. So following the advise of Eric J. Evans in his book Domain-Driven Design: Tackling Complexity in the Hart of Software we noted for ourselves following parts that should be presented in our Domain Model in order to consider it us a separate layer:

  • Entities accompanied with business methods
  • Value objects
  • Repositories
  • Aggregates
  • Bounded Context

By analyzing our model designed for Android or iOS platform and for our back-end platform we realized that only our back-end model meets Domain Model criteria and deserves a separate layer but in case of mobile phone platforms we ended up with simple entities that were the part of application or Use Cases layer.

By modeling entities we replicate real life entity attributes, like person’s name, surname or gender that are operated by methods encapsulated in the same entity or in service. At the end of the modeling process we come up with our business model that further can be converted to the Entity-Relationship model, that is used to build the database. In such way Domain model or plain entities are tightly bound to normalized ER model. And one can live only with one ER model until the moment when displayed information starts more and more deviate from the ER model and in order to fetch the data from database more complex queries are required. And this is the time when we started to consider CQRS. But now it was not clear how to integrate the CQRS in Clean Architecture layers and we turn to the mighty Google for someone’s else experience and we found this wonderful talk that showed us exactly what we were looking for and so we ended up by extending the application layer with commands and queries and adding separate Database context or our queries.

Please visit our website and join the mailing list. Our app is coming soon:

http://fitradar.me/

Keeping Write and Read operations separately

In this article I wanted to share our team experience on how we arrived at particular architecture on our back-end solution. On the back-end we wanted to work with technologies that our team is familiar with and had an experience with before but at the same time is not outdated and has a great potential in the near future. In our case it was ASP.NET Core framework. I am not going in to the endless discussion about what framework or language is the best, from my point of view it is useless since such a topic is very biased. And I already mentioned in one of my previous blogs that despite the fact that the software development supposed to be an exact science, and many aspects of it really is, the choice of language, frameworks and best practices many times is just a matter of personal preference – something that you feel more comfortable with and it doesn’t have any scientific justification. And when I will be laying down the arguments for the solution we came up with it will be presented from our team’s point of view and how it helped us to make a design and implementation more clear and easier to understand, which might not be the case for other teams.

ASP.NET Core comes comes with some prepacked architecture that satisfied our needs. For REST full Web API solution MVC is very suitable design pattern and gave us a good starting point. Dependency injection in turn allowed us easy start to write unit tests and mock dependencies. But the more complex solution became the more we saw that we need to extend our initial project with more general architecture, after all MVC is just an UI level design pattern. This wasn’t the first project in our team’s experience when we had to split application’s code in layers. Some years ago very common was 3 tier architecture:

Three tier Application

And we used this architecture quite a lot. In some projects we felt it suited our needs perfectly, but in some we had a feeling that we do some kind of workaround to fit the architecture. For some time I could not really even explain why those projects didn’t comply with the above architecture until the moment when I read about Command and Query Responsibility Segregation (CQRS) pattern. It was one of those aha moments when you discover what was really bothering you all this time. In the traditional 3 tier architecture the same data model and the same database is used for read and write operations. It works well when we need to display the same information we saved earlier. But the more complex application becomes the more a model we use to save data starts to deviate from a model we use to display data. For example most of the applications today require a user to create a profile or an account. Let’s say we save this information in table User and in order to do that we use an model with the same name. In case we want to see our account information we will fetch the data from the same table User. But this is only one use case where we are displaying the information about user. In real world applications information about user is displayed in many other pages together with other information, for example in e-store that would be information about product and product category, in blog that would be a post and so on. And in order to receive data suitable for a view complex queries with joins and sub-queries are used. And not only models for read and write operations are different but the requirements for those operations are different as well. In case of insert update and delete operations the database should be normalized that allows us to minimize duplicate data and avoid data modification issues. The database normalization usually results in more tables than initial design. The query operations on the other hand are focused more on performance, that can be improved by denormalizing tables. The beauty of CQRS is that it allows to separate the write and read flows by using different models and even different databases. In last case it would allow to scale the databases for read and write operations independently. And at some point of time this feature may become very crucial since write operations are significantly less that queries. As you can see there are several levels how far we can separate commands (insert, update, delete operations) from queries (read operations):

  • The lowest level of read and write operation separation is on Repository level – we are using the same Domain model for write operations and for display data in UI, but we have a separate method in Repository for querying database. In order to display only necessary data we have to introduce View model and map data from Domain model to View model.
  • Next level of CQRS maps View Models directly to database queries. There is no need for mapping between Domain model and View model anymore. On database level we still have normalized tables that correspond to Domain model. View models in this case correspond to queries that involves joins and sub queries.
  • The two above levels of separation sometimes are not regarded as real CQRS pattern but are known as CQS (Command Query Separation) pattern, therefore only the next level of separation when write and read operations are regarded as two separate workflows throughout an application is considered as CQRS pattern.
  • The final level of separation goes further even to the database layer, where each type of operations interact with its own database

In our back end API solution we decided to separate write and read operation on the application level and not to use a separate storage, but instead we are using SQL Views for queries. It allows us avoid data synchronization between the databases. But at the same time architecture is opened for further extension and possibility to add separate storage for queries. And so we ended up with following architecture:

Visit our website and join the mailing list. Our app is coming soon:

http://fitradar.me/

Bringing security in FitRadar solution

In this article it is time to talk about how we secure our mobile application and our back-end API. Most of the information displayed in Fitradar mobile application is user dependent. Starting with user profile that has unique information for each user and can be changed or deleted only by the owner of the profile and ending with sport events map and timeline where information is built based on user preferences. And we have to provide access to third party integration services like Firebase Storage and payment gateway. As you can imagine in order to allow our application users to store and access personal information in a secure way we needed to implement user authentication and authorization.

It was clear from the very beginning that we are not going to develop our own authentication service but instead we will use third party solution. And before we started to explore available solutions we laid down following requirements:

  • the access to Fitradar Web API should be granted only to authenticated users
  • the access to Fitradar mobile application should be granted only to authenticated users
  • the access to Fitradar Web API should have different privilege levels
  • the access to third party resources like Firebase storage should be granted from the same authorization service
  • the sign up and sign in pages should be part of the Fitradar app

After some investigation we came to conclusion that combination of OAuth2 and OpenID Connect protocols is the best solution for our needs since it:

  • provide a mechanism for our resource and third party resource protection
  • authenticate users using local or remote account store

Once we were clear about the authentication flow and protocols we started to look for the OAuth2 and OpenID Connect protocol implementation providers. First we wanted to have as much as possible control over authorization service, because we didn’t want to land in situation where authorization service would restrict our application look and functionality. For example RFC8252 (OAuth 2.0 for Native Apps) states that: “OAuth 2.0 authorization requests from native apps should only be made through external user-agents, primarily the user’s browser.” And that might enforce our app to use authorization server sign in and sign up user interface. And since on our back-end we are using ASP.NET Core we decided to use IdentityServer. For a while it worked quite well, but then we started to noticed that there are few aspects of the OAuth2 protocol that we have to implement by ourselves, like Access token lifetime in our mobile app. So we started to feel that we are spending too much time on implementing and maintaining the protocol features that we were quite sure should be working out of the box. Although IdentityServer offers full fledge OAuth 2.0 and OpenID Connect implementation but we still had to host it on our environment and maintain it by ourselves. And the maintenance question bothered us the most. For the startup company with limited human resources to have a solution that might require an administration seemed for us a high risk. If something goes wrong with authentication we will have to put all our effort to fix it, which means the other work will suffer from it. So we decided to look for a cloud solution that would free us from the maintenance burden. And once again we searched for available authentication and authorization providers but this time on a cloud. And after a while we came up with two potential providers: Firebase Authentication and Azure Active Directory. First Active Directory seemed a good solution for our needs:

  • very well established user management system (although we just needed a tiny portion of all available feature)
  • good documentation and code samples
  • very easy integration with .Net Core
  • possibility to use custom sign up and sign in pages

Although Azure AD integrated very well with our Web API and there are good sample projects how to use it with Android and iOS applications we were not sure how well it will integrate with Firabase Storage that we are using to store user images. It turns out we can grant the access to the Firebase storage resources only to Firabase users. To integrate with other OAuth providers Firabase creates a new user account after user has signed in for the first time and links it to the credentials. The fact that we would have user accounts on two authorization servers that we have control over really held us back from integrating Azure Active Directory B2C in our solution. From the other hand we were hesitant to start to use Firebase Authentication service in our ASP.NET Core solution as well, since we were not sure how much time and effort it will require from our team. But after all the Firebase Authentication is just another OAuth 2.0 and OpenID Connect provider that issues identity and access tokens and Jwt bearer authentication middleware in ASP.NET Core application can validate those tokens and authenticate a request. So we decided to spend some time to create a proof of concept project that would show us how much time we will have to invest in order to integrate Firebase Authentication in our Web API authentication solution. And it turns out that requires just a few lines of code in Startup.cs file:

services
    .AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer(options => {
        options.Authority = "https://securetoken.google.com/fitradar-firebase-project";
        options.TokenValidationParameters = new TokenValidationParameters
        {
            ValidateIssuer = true,
            ValidIssuer = "https://securetoken.google.com/fitradar-firebase-project",
            ValidateAudience = true,
            ValidAudience = "fitradar-firebase-project",
            ValidateLifetime = true
        };
});

And bellow is the final solution we are using to secure our and third party resources and authenticate a user

Please visit our website and join the mailing list. Our app is coming soon:

http://fitradar.me/

Bringing order in files

One of the main paradigm we followed during Fitradar application development was Object Oriented Programming paradigm. And the main objectives of OOP are:

  • to organize the code in such a way that data and functions using its data stick together in one entity (class),
  • to extract reusable parts in separate entities (classes interfaces).

And as we followed the OOP principles and patterns our big code evolved in to many small files, each representing one or sometimes several entities. Each file contained clean and well organized code that was easy to maintain. From one hand we reduced the size of the files and such improved the navigation within a file but increased the number of files. And the more files we produced the more harder it became to navigate between the files. And very quickly it became clear that we need a new way how to organize our code-base files that anyone could quickly find a needed file. And since there are several ways how to organize the files in packages and the source code packaging really depends on the project, in this article I wanted to share our teams experience on how we found a way that helped us to find file quicker in our code base.

The goal of organizing files in packages is to allow a developer or any other person who is working with a source code easily find a needed data type. In order to achieve this we had to introduce particular principles on how to organize files within a packages. And once a person learns these principles it should be a breeze for him to find a necessary type. When we thought about it, we came to conclusion that these principles should act like search algorithm but for human. The basic partition of our source code in separate projects was predefined by Clean Architecture. It gave us a basic understanding where to put files on the high level. In our first attempt we tried to put the same type data under the same package. For example all the repositories definitions we kept in package com.fitradarlab.fitradar.domain.repository, all the retrofit endpoint definitions we kept in package com.fitradarlab.fitradar.data.net.endpoints and so on. This kind of approach introduced by Clean Architecture worked well in data and domain projects, but when we tied to apply it to the UI project it didn’t really helped us. And the reason was the way how we worked with UI part of the project. Our work was organized around the use cases. And to implement a use case on the UI level we had to work simultaneously on Activity, Fragment, ViewModel, Dagger dependency Module, layout and navigation. All these types were located in different packages and under each package there were already quite a few other files and therefore it was hard to find a needed file fast. First to mitigate the problem we tried to keep all the files of a use case opened, but we realized quickly that the more files we open in Android Studio the less we see of a file name in a tab because it shrinks. So even on our big screens we could have only 5-7 files opened, but in many cases we needed more than that. It was not right away that we noticed that the files we try to keep opened belong to one use case, but once we realized that it became clear that we need to put those files under the same package. Once this discovery was made the new packaging structure for UI project was born. We completely refactored UI project by introducing packages that reassemble the names of our use cases. And thanks to Android Studio refactoring tools it took only a few hours, and after that we really felt comfortable with the new packaging structure. Now we didn’t have to keep the bunch of file opened because all the files we needed to work with were visible under the single package in Project window.

New UI project package structure

But there was still one problem left – the resources files. Contrary to source code where developer can create a hierarchy of packages the resources have only several predefined folders and the most used resource types like layouts and drawables usually have long list of files. And once again we applied the use case approach and came up with following naming convention for our layout files: the layout file starts with the type – fragment, activity, row or view, then we mimic the name of the package and the name ends with unique name of the layout. For example the layout for our timeline page has the name fragment_sport_event_timeline.xml. Unfortunately we still can’t find a good naming strategy for drawables and other shared resources that are not bind to particular use case, but already now with these new naming conventions we see a noticeable improvement in our source code maintenance.

Visit our website and join the mailing list. Our app is coming soon:

http://fitradar.me/

Collaboration between developers and non-developers in FitRadar

Initially when we worked on the idea of Fitradar several people were involved – among them were sports club manager, designer, and developer. Everyone put his ideas on the table and after several brainstorm meetings we came up with initial requirements for our future product. Later all who participated in user requirement creation became in one or another way a tester. The user requirement authors knew the best how the app should work, so they were the ones who could verify the application’s design and the functionality.

Now when we have a tangible product, the people who laid down the requirements want to make sure their ideas are working properly in the application. Some of the features are possible to test via User Interface, like design and navigation. And in this case a screen is serving as a medium of communication between developers and testers. People who are testing the application can take screenshots and explain the problem to developer in a demonstrable way. But how to make sure the business logic implemented in application is behaving according to the requirements. Developers implemented it according to their interpretation of requirements. Although we used UML diagrams tables and pictures, many times the only way to describe the business logic was a human language. And as we know human language is quite ambiguous and fills the gaps with a lot of assumptions and that inevitably leads to misinterpretation. One way for non developer to test the applications business logic is to try to invoke it from a UI and see if it works correctly, but developers and QA know that in such way we are covering just a small portion of test cases. So there must be a better way for user requirement authors, would it be Quality Assure, Business Analysist or any other team member, to make sure the business logic he proposed is functioning as expected.

I already wrote about the test strategy in our team and well known test pyramid. The lower level tests are written by developers and only time when they need an input from other team members is the time when they are interpreting the requirements. Once the requirements are clear they can write the code and accompanying unit tests. Even UI tests can be written with little help from non developers, but the part where user requirement authors start to play important role is functional tests. In case of our app the UI is separated from the rest of application and most of the business logic happens on the back-end. So the functional tests in our team include back-end API tests and mobile application API tests.

The way how we helped the non developers in our team understand the implanted business logic was by using some principles of the Domain Driven Design and Behavior Driven Development. I should say not everything what Eric Evans described in his book “Domain Language Tackling Complexity in the Heart of Software” was applicable to our solution, but some aspects really helped us to establish a good way of collaboration between team members and today I wanted to share those aspect of DDD:

  • Ubiquitous Language, the term introduces by Eric Evans, helped our developers and the rest of the team speak the same language. The main idea of Ubiquitous Language is to create a common vocabulary of a given business domain that both domain experts and developers would speak. It was quite easy to establish since in the process of creation of user requirements both developers and non developers were present and developers knew quite well the business domain. Most of the value of the common language we started to see when developers had to read their implemented logic to non developers
  • The next step when the vocabulary of app was created, was to create the API methods that non developers are able to understand and follow. In order to achieve this API methods were written on such abstraction level that only domain vocabulary was used – all the technical details were hidden in the underlying classes and methods. Even logical operations were converted in to the methods that expressed the domain concept.
  • Once the API methods were in place, the business logic authors could participate in API review. But even then there might be a flaw in the requirements itself. And so we needed to write tests expressed in domain language. For this purpose we use some aspects of Behavior driven development, particularly we used Cucumber language (Gherkin plugin on Android Studio and SpecFlow extension on Visual Studio) for writing functional and UI tests. Since the Cucumber language is intended to be used not only by developers but by regular humans as well then everyone who contributed to our application’s user requirements was able to add tests as well.

Visit our website and join the mailing list. Our app is coming soon: http://fitradar.me/

Logging in our application

Any application to some extent is using the logging system. And we are not an exception. We log a lot of information in our Android and iOS applications as well as in our backend server application. But what logs give us, after all, they are not contributing to the application functionality? They might be a part of user requirements, but why would anyone want to invest money and time for the feature that is not used by a user? Let’s explore the purpose of logs in software development and practices developers are applying.

Here are some of the cases where we are using logging in our applications:

  • during development to make it easier and faster spot a bug along the
    • @NonNull and other annotations and code inspection tool Lint
    • unit and integration tests, where we try to cover most of the code execution paths and integrations points. (And I already wrote how we approached the testing in our project)

    we use logs that show as the trace of actions and values that led to a particular bug. But before we started to use logs we had to answer the following questions:

    • where to put log statements on
    • what information to log

    Since a lot of our code is covered by unit tests then there is little reason to log our own code for debugging and test purposes, but where we don’t have control over the values we are processing is:

    • framework lifecycle methods, like onCreate, on ViewCreate, onResume, onPause in Android application. If our code is using the arguments provided by lifecycle method we log these values and assert the expected value if it is possible
    • to click listeners and other event handlers
    • third party library methods and their returned results. For example, for network calls we are using Retrofit and we always log the network requests and responses in debug build variant. Another example calls to the database. Since we are using Room in Android application, we want to see SQL code sent to the database

    So once we log enough information we can spot the bug just by going through logs and skipping the process of starting the app again with attached debugger and stepping through the code. And we don’t have to worry much about the performance as well since Android logging system is taking care of debug log statements and skip them at runtime.

  • in production:
    • to monitor, on the back end server the logs are used to monitor the normal business logic functionality. We log every request. If the request is successful we log it as info, in case request ends up we log error. When the error is reported on the back end service, the gathered logs are of great value. Very often they allow to spot the problem without further investigation.

    • we log fatal errors, that caused the application stop. On the backend server, we catch the exceptions as early as possible and when the exception is caught it is logged and an email is sent to an administrator. We don’t experience many exceptions outside of our app on the back end part, because the only reason that can lead to such exception is a malformed request and not the back end code itself, and since the request is formed by third party library it is very rear that it is malformed. A different story is with uncaught exceptions on the client – Android or iOS app. It is not possible to catch all the exceptions so they leak and cause the app to crash. To log these kinds of exceptions we use Fabric Crashlytics (Firebase). These logs later help us a lot to find a cause of a crash.

Visit our website and join the mailing list! Our app is coming soon:

http://fitradar.me/