Evolution of Core Integration Framework
From very beginning our company developed software solutions for small and medium clients. Besides development, we support and grow our products based on a new customer requirements. At some point we realized, that all products we develop share common core functionality, that was rewritten over and over using similar libraries and methodologies. This led us to develop Core Framework, which became basis for most of our new projects. Every product we developed required some sort of back-end and configuration UI. Shared components include:
- User management
- Role and permission management
- Client management
- Authentication and authorization
- Running tasks on schedule
Having those components ready drastically reduces time to market for new products.
At that moment we invested our effort into identifying technologies that would allow us to build a development platform. The requirements included UI/UX, back-end, database and interface between back and front ends. As for back-end we naturally chose Java as it was cross-platform product with very rich ecosystem of frameworks and libraries. Choosing database was a bit tricky. Many of our projects required different DB engines, such as MySQL, Oracle, Informix, PostgreSQL and many more. We choose Hibernate ORM as interface layer between application and database, as it uniformly support most SQL variants.
We evaluated many UI/UX frameworks. Since our platform targets mostly administrative interfaces, main requirement was richness and flexibility of UI components. Out of many options, such as Sencha, Dojo, JQuery UI and other we selected DHTMLX framework. It had very extensive documentation and robust examples. Ability to start using components without steep learning curve was one of the most important criteria. API was consistent and it was fairly easy to integrate different components into layouts.
Having a robust technology stack containing Java, Hibernate, Spring Framework, Hessian, DWR and DHTMLX we were able to combine all this together into Core Framework.
We had to extend DHTMLX and add some wrappers around components to reduce boilerplate code and allow them easily integrate with our back-end via DWR calls. In the end we got very powerful and easy to extend UI framework that is easy enough to use for standard cases and extensible enough to build complex single page interfaces.
As user interface was evolving, we ended up using 2U layout a lot. Where on the left side we have a list of entities and details of selected entity on the right. Unlike regular web pages on websites, administrative interfaces are better fit in one screen and user should be able to view all necessary information at once. When there is not enough real estate for all the details to fit into the right pane without scrolling we use tabs which allow us logically divide selected object’s properties and display only relevant information.
DHTMLX grid component provide powerful support for infinite scrolling. Thanks to this functionality we can effectively present data in searchable and scrollable lists with billions of rows.
Thanks to robust form validation in DHTMLX user can clearly see what is missing or has invalid data.
Many of our clients are located in non-English speaking parts of the world. Core Framework was designed from ground up to support multi-lingual interfaces. Administrators can select default language and specify additional active languages.
Each user can select preferred language which will be used in the interface when user logs in.
Not only labels and menus are localized but data could be entered in multiple languages as well. This could be seen in Contact Types example below.
Interface and data localization is a very tedious job, requiring matter experts in specific language. In some cases time to market is critical and translation cannot be done in time. For such scenarios Core Framework provides integration with Google Translation cloud services. When cloud translation is enabled, interface will automatically call this service to translate missing sentences to target language. Otherwise text in default language will be presented.
Unfortunately, DHTMLX framework does not support RTL languages, and hence we are unable to provide interface localization for Arabic, Azeri, Dhivehi/Maldivian, Hebrew, Kurdish (Sorani), Persian/Farsi and Urdu. The Central Semitic language alone is used by about 1.7 billion people most of whom are in the Arab world. This is huge market that we would like to enter. We strongly encourage DHTMLX developers to consider including RTL support into their products.
Scheduled tasks support allow products based on Core Framework to execute tasks at specific times.
Historical overview of scheduled tasks allow administrator to view tasks side by side. DHTMLX scheduler allow to displaying tasks in such grid. Users are able to analyze and tune long running tasks, move execution time to limit impact of heavy tasks on other system’s functionality.
Existing products are tend to evolve and as new requirements arise we need to add more functionality into the systems. Adding a new module usually require system restart. Some of our products were deployed in an environments where restarts could be either undesirable if it runs 24/7 or maintenance windows are scheduled at very inconvenient times. After analyzing change requests across many products we realized, that besides bug fixes, most new functionality requests involving adding something to UI screen, change localization or develop a new scheduled task.
Changing localization messages was a bit more involved. Initially we had all our localized messages in text files embedded into WAR archive. We moved all the messages into database and created service that reloads messages on demand. By doing so we got rid of redeploys related to localization changes.
Core scripts are not necessarily executable. Core Framework uses Velocity scripts to generate notification emails in HTML and in user specified language. Some complex SQL queries could be stored as scripts and modified on the fly.
Initially, we had our products running on one machine. In many cases we provided an administrative UI for much larger systems. This UI was storing data into shared database. For most applications it was enough, as downtime due to system crash or reboot was not critical. It was ok for admin page to be unavailable for couple of minutes. But we were growing and our projects were becoming more and more complex and demanding. We began to deliver complete solutions to larger clients. And our clients were stressing to have high availability and scalability in the products we deliver.
Next blog post will describe clustering architecture of Core Framework and pitfalls associated with developing distributed systems.