It is a long held belief amongst software developers that software must have the ability to dynamical handle as many edge cases as possible to be useful. This is true, a perfect workflow can be created for a standard case but any developer will tell you that user input is never exactly what developers expect it to be. This is why developers have to create systems that allow their software to interpret as many of these edge cases as possible.
The Next Data team had to consider this when developing the engine, both in the edge cases our community will have to develop for, and the edge cases that our community will create themselves. Essentially every application made using the engine is its own edge case, and the engine needs to be able to handle these; there is no point in developing an engine that can only handle the software problems that we could think of. This is why the Next Data Engine was created with a components system, which allows developers to ‘plug’ their own edge-case code into the engine and allows developers to more easily create robust solutions to handle the edge-case requirements of their users.
The components system in the Next Data Engine functions by providing a large amount of generic functionality that is understood by the engine itself, from which developers can create their own logic to implement the generic concepts for their specific circumstances.
A good analogy for the component system is thinking of the Next Data Engine as a breadboard, it provides the power, some the basic design of what a resistor, LED, capacitor and transistor are, and the ability to start plugging components together to form systems. You can create resistors with higher resistance values, or that are light sensitive, and plug them into the breadboard to form your solution. In the same way the Next Data Engine provides the basic design of what a file system component, database connection, job component, etc. are and gives developers the ability to start plugging these components together. You might make a file system component that links your solution to Dropbox, or maybe you will create version that takes another file system as a parameter and automatically creates backups there. What ever logic you create, it can be plugged into the engine and other components using the engines components feature.
For example, the Next Data Engine supplies a base file system component, which provides common file system logic such as file paths, deleting and creating files and checking if files exists. The specific implementation of these methods will be significantly different between, for instance, physical file systems and a Dropbox file system. Developers have therefore been given the ability to write their own implementations of the base file system, telling the file system how it is to complete the functions such as creating and deleting files for this specific file system type.
This type of abstraction will be fairly familiar for experienced developers, what makes the Next Data Engine’s method of components special is their ability to be instanced and stored in the engine directory with parameters. Going back to the file system example from before, specifically for a Dropbox, you will need some form of identification when connecting to the server, so when writing the implementation you may add a username and password parameter to the component. You can then create an instance of the Dropbox file system component with a given username and password, and then a second (third, fourth, etc) instance of the component with different usernames and passwords. Each of these Dropbox file system components can then be passed as a component parameter into another component, for instance a job component, as the file storage for whatever functionality this other component provides. The simple diagram to the right shows how the concept could be applied to generate a report from a database and save it to a file system.
As you can see, the implementation of any given piece of functionality is abstracted from other individual pieces of functionality, the generate report job is only aware of the fact that it needs to save a file to a file system rather than knowing what kind of storage it needs to talk to. This makes it extremely easy to quickly develop new solutions for edge cases. If suddenly you need to transition from Dropbox to OneDrive, then simply plug the OneDrive component into the existing job without even opening the code for the job component.
It is worth noting that while you are completely able to write the implementation for your own components, there are many packages available for free on our package store that provide the implementation for common circumstances, such as those discussed so far in this article. Check out what components are available for download here: LINK*****
In conclusion, the components system of the Next Data Engine is a powerful tool for creating the kinds of modular applications that allow edge cases to be handled very efficiently. By leveraging the power of this system you organisation will be well on your way to creating dynamic applications with the ability to be configured for many situations in no time!