The ability to author complex formulas is both a blessing and curse of the iFormBuilder platform. Using JavaScript to write Smart Controls is akin to playing in a sandbox, letting users experiment and discover novel strategies for optimizing their individual data collection.
Over the past decade, we too have learned and adapted the recommended best practices for writing Smart Controls. At the heart of these best practices — and any Smart Control statement — is properly referencing other elements.
If you are a longtime user then you’ve undoubtedly heard phrases like “data caching” or “clearing data”. We’ve learned over time that these techniques were masking a symptom rather than addressing the underlying architecture of the software and embracing how the application functions.
The purpose of this article is to dispel outdated form building strategies and promote new best practices. In order to do so, we’ll outline how data is stored in the iForm app, several app-specific idiosyncrasies that outdated strategies lean on, and our recommended best practices for future form building.
How is Data Stored?
Understanding the structure and availability of data is critical to properly writing formulas and JavaScript Smart Controls. While it is common knowledge that element values are referenced using their data column name (DCN), that is just scratching the surface of how data is actually structured.
For every DCN, a variable of equal name is created. This variable is an identifier that gets/sets the value in memory. The variables are free-floating, meaning they are not tied to a specific form. If two forms (A and B) both have a field called first_name, then that variable is shared/overwritten as each form is used.
What is less known is that each form also has its own variable equal to the form name. Each element of the form is stored within this parent variable name. Even a subform is stored within the top-most parent variable, creating a structure akin to a folder directory on your computer.
This can be hard to mentally grasp, so let’s do it visually. If I have two very similar forms, one called contractors and one called employees, the following diagram illustrates the variables and data structure within the app.
There are six top-level variables here: contractor, employee, first_name, last_name, email, and ssn. So, how is the data model affected as we use the app? Let’s go through the process of collecting some data and track the changes in memory.
We’ll start by filling out the employee form. After filling out the form and completing the record, the memory would look something like this.
Notice how the values are stored in both the form-specific fields as well as the free-floating variables? Well, let’s see what happens when we fill out the contractor form. Upon completion, the memory would look something like this.
The fields in employee stay untouched; however, the free-floating variables are overwritten because they are shared resources. Additionally, the old ssn value is still present. The important takeaway here is that free-floating variables are never cleared, only set or overwritten.
So now what happens when we go to fill out a form for a second time? When we first open the form, the free-floating variables persist, but the form-specific fields are reset.
Each field is set/overwritten as we fill out the form. Let’s say we fill out the first name and last name, a snapshot of the memory before entering the ssn would look like this.
We’ll discuss the repercussions of this data architecture later, but for now the important takeaways are as follows:
- There are two types of variables: form-specific and free-floating
- Free-floating variables are never cleared, only set and overwritten
- Form-specific variables are cleared when a blank form is opened
- The free-floating variable space can get messy and disorganized
Dispelling the Misconception of Caching and Clearing Values
References to “clearing values” and “data caching” can be found throughout much of our documentation and previous training material. We speak/write it on a daily basis at the Zerion office.
The reason for this is because the vast majority of the discourse only focuses on the free-floating variables. Simply put, they are easier to use and shorter to reference, making great examples for training.
Unfortunately, this has led to a notion that we need to reset values to avoid discrepancies in data. For those unfamiliar with these inconsistencies, let’s return to the memory state from above.
What happens if we have a formula field to store the text “last_name, first_name, ssn” for future use in a barcode label? Writing that using the free-floating variables, the Dynamic Value would look something like this.
last_name + “, ” + first_name + “, ” + ssn
The problem is that we have Maurise’s ssn stored. We refer to this as “caching” because of a value from a previous record being carried over. To resolve this caching issue, we’ve recommended “clearing values” by setting a Dynamic Value such as ssn = “”
While that strategy may appear to fix the problem, all we’ve done is apply a band-aid because the Dynamic Value is supposed to represent a formula to populate the element. We don’t actually want an empty string stored, we just don’t want the old value.
The practice of clearing values to avoid caching is so pervasive that we even have an article documenting how to clear each data type. Furthermore, this brings up another deficiency. There is no way to truly “clear data”.
For those with programming experience, the term null is used to represent the absence of a value. Simply put, iFormBuilder does not allow null values to be set. Using an empty string, a zero, or a -1 in the Dynamic Value does not actually set a value to null; these are all proxies to be interpreted as null.
The behavior is more apparent when working with elements such as Date, Time, or Date-Time which cannot be cleared. This is not a UI-oversight, it is an intentional constraint because the underlying value cannot be cleared itself. Since we cannot set elements to null, we must avoid giving any element a value unless it is necessary and inline with the actual data that is collected.
To summarize, best practices involving clearing values to avoid caching are outdated. At best they are a band-aid for improper references, at worst they can lead to corrupted data. Like the above section, here are some simple bullets to internalize:
- Cached values are a manifestation of the free-floating variables
- Formulas for “empty values” should not be needed
- Using empty strings for text, zero for numeric, and -1 for option-based elements are nothing more than proxies for a null value
- True null cannot be set via formula or UI
- Once a field has a value, that value cannot be deleted since null cannot be set
How to Properly Reference Elements
So far we’ve explained how data is stored and what not to do. If you’re a long-time form builder then I’m sure you’re thinking: I’ve been building forms this way for years, what should I be doing instead?
Fortunately we have the answer. To completely eliminate caching and the need to “clear values”, elements should always be referenced by their full form path. That is, the form-specific variables should be used and not the free-floating ones.
Let’s return to our label for barcode. Using the full form path, the Dynamic Value would become:
employee.last_name + “, ” + employee.first_name + “, ” + employee.ssn
The period or “dot notation” is used to reference nested variables. This can be read as within the employee object, reference the field last_name. By using the full form path, we take advantage of the fact that a fresh object is created when a new record is created. Thus, there will never be caching issues as long as you reference within the form object.
In truth, this is not a new concept. If we look at the documentation for the iformbuilder.math() built-in function, we see that a reference using the full form path is used.
Conclusion
The required mindset shift is that the full form path is the de facto standard and not the outlier or exception.
This shift in implementation has implications for the design and maintenance of your forms. Since the full form path is used for references, subforms with Smart Controls and option lists with Condition Values will be limited in their ability to be reused. Additionally, referencing elements nested within layers of subforms becomes exceedingly long and easy to mess up.
We believe the integrity of data is paramount above all else, which is why we are advocating the following best practices:
- Avoid “clearing values” using empty strings, zero, or -1
- Always use the full form path to reference values
Building in a sandbox environment like iFormBuilder can be challenging, but understanding the rules that govern data storage and access is the first step in harnessing the power of the platform.
If you have questions about your existing forms or strategies for a new idea, please reach out to our team at support@zerionsoftware.com. Happy building!
