Creating Upgradable Templates¶
Creating Upgrade and Downgrade Paths for Switching between Versions of an iApp Template¶
Introduction¶
One of the most powerful features of iApp templates is the ability to
copy them, and make new versions with new capabilities. An existing
Application Service can be re-parented to a new iApp template to take
advantage of new features (re-parenting is simply clicking the change
button on the template row while in the Reconfigure menu). However, if
there are too many changes in the new template, it can become
incompatible with the old template, and problems arise during
re-parenting. Re-parenting works when the section and input names
remain the same between versions. If these names change, the data
associated with the changed inputs are lost upon re-parenting. In
cases like this, the user is forced to re-enter information that
existed in the original template after re-parenting. If the changes
were severe enough, re-parenting is not possible at all, and the
deployment must be rebuilt from scratch using the new template. In the
past, this issue has been the source of considerable difficulty, but
the F5 Product Management Engineering (PME) team has developed a way
to solve these sorts of problems.
Beginning in BIG-IP version 11.4.0, the iApp templates that ship with
the BIG-IP system will be able to upgrade an iApp application service
from the version of the template on which it was created, to the new
version; despite the fact that the new templates in 11.4 are
significantly different. This document explains how this new
re-parenting/upgrade process works, and how to accomplish it within
the iApp framework.
In version 11.4.0, there will be an upgrade and downgrade procedure
available on the BIG-IP system that helps users with this process. The
upgrade procedure (and the data fed into it) enables an existing iApp
service to work with a new version of the template. The downgrade
procedure allows a path back to the legacy template, in case users are
not satisfied with the new one. If you want to build a template that
is able to perform these upgrades and downgrades, and you need it to
work on version prior to 11.4.0, you must include the procedures in
Appendix A and Appendix B in the template.
How does it work?¶
When a user submits an iApp template, the answers to all of the questions that are visible at the time of submission are stored in variables in an iApp Application Service Object (ASO). These variables are presented to the implementation section, and are also used to re-populate the fields when reentering a template. The variables inside an ASO should be familiar to developers using them in the implementation section. These are the variables that take the form
<section name>__<input name>
. In order to make the ASO that was deployed with the old version of a
template compatible with the new version, the variable names need to
be changed to map to the new section and input names. In many cases
the answers need to be transformed as well. This is accomplished by
defining all of the required transformations in array-based data
structures which can be passed to the upgrade and downgrade
procedures.
The upgrade procedure takes two arrays as parameters: the upgrade
array and the translations array. The upgrade array provides mappings
between the input fields from the old template to those of the new
template. The upgrade array allows moving inputs between sections, or
otherwise renaming them, and still preserves the data from the old
template into the new one. For example, if the old template contains
an input named “Foo” in a section named “Intro”, the upgrade array can
be used to map that value to a different section in the new template.
The translation array creates translations for input values. This
allows the values in drop-down lists to be changed between versions.
Without the translation array, if there was a choice input in the old
template with possible values of ‘No’ and ‘Yes’ and the possible
values for that input were changed to something more descriptive in
the new template, such as ‘enabled’ and ‘disabled’, there would be a
problem after re-parenting. The value would be moved forward
correctly, but the value stored in the ASO would be out of step with
the input and with what the back-end code is expecting. The
translations array allows translating the values for the inputs to
their new values, so an answer of ‘Yes’ can be changed to ‘enabled’
during the upgrade and continues to work in the new version.
The downgrade procedure takes a single array as a parameter. The
downgrade array defines which tables in the new template correspond to
which tables in the older template. All of the values for the old
template, except for the tables, are preserved in the ASO. This means
that as long as the tables are reproduced from their corresponding
table in the new template during downgrade, it is possible to get back
to the old template.
Workflow¶
Upgrading a template requires two steps. First, there is one round of
submitting the template to perform the upgrade of all of the ASO’s
values to their new values. And then the user has to reenter the
template and submit it again order to run the implementation
section with the new values and change the configuration accordingly.
Re-Parenting vs. In-Place upgrades¶
The PME team investigated two different approaches to doing upgrades,
called the Separate Template method and the Converged Template method.
They both have advantages (described below), but in the end, the
Converged template method was chosen because it was the cleaner of the
two, especially considering the issues associated with users upgrading
their BIG-IP systems from a previous version to the new one. Separate
Template method
With the Separate Template method, both the old version and the new
version of the template exist on the same box with different names.
The new version of the template contains the upgrade capabilities.
When a user wants to do an upgrade, they enter the new template,
select upgrade, and then the user is able to select an iApp service
from a from a drop-down list. When the user submits the template, the
values are pulled from the selected ASO, run through the upgrade
procedure, and stored in the new ASO. When the user reenters the
template, they see it populated with the values from the old template.
They can submit again and get their new configuration.
This method is the most straightforward, and makes a lot of sense for
certain situations, but there are some real drawbacks. The most
difficult is the fact that the old configuration still exists. The old
ASO isn’t replaced; its values are just copied and translated into a
new ASO. This means that the old ASO must be deleted or the virtual
server IP addresses on one of the deployments must be changed.
Otherwise, there would be a virtual server IP address conflict on
submission. This lack of flexibility was ultimately the largest factor
in the choice to use the more involved Converged Template method.
Converged Template method¶
With the Converged Template method, both the old version and the new
version of the template exist in the same template, but only one is
visible and accessible at any point in time. If a user starts a new
deployment with the new template, they only see the new version of the
template. However, if a user migrates an ASO to this new template (or
they upgrade their BIG-IP version and get a new template attached to
their ASO that way) then they will see the old version of the template
UI when opening the template. The user is shown a message saying that
the version they just opened has been deprecated, and they are offered
the option of performing an upgrade. If the user doesn’t want to
upgrade, they can continue to work with the deprecated template. If
they choose upgrade, the ASO is upgraded to work with the new version
of the template. When the user next enters the template, they see the
new version of the template with the values carried over. To complete
the upgrade and update the configuration, the user simply submits the
template. The option to downgrade back to the old version of the
template is always available, in case the new version does not meet
the needs of the user.
This method is more work and makes for large templates; all of the
code for both versions exists in the template, along with the upgrade
and downgrade arrays. Also, some special coding is required to make it
all work, especially for the template to auto-detect and show the
correct version. Even so, this method enables a much easier to use and
more flexible solution, and as previously mentioned, was the choice of
the PME team. This is the method that is described in the remainder of
this document.
Implementation¶
In the following sections, we describe how to combine the old and new
versions of the template into a single template, how to make the
auto-detection work correctly, and how to build the upgrade,
downgrade, and translation arrays. The exact way you go about doing
this depends very much on the nature of the templates and how
extensive the changes are between versions.
The PME team uses the following steps to create converged, upgradable
templates:
1. Develop and test the new version of the template in its own
template file.
2. Analyze how to converge the old and new templates into a single
file and do so
3. Write the upgrade, translation, and downgrade arrays.
4. Test the upgrade and downgrade paths
Building an Upgrade Array¶
The upgrade array maps values from one input to another. If the array
is built using array set notation and two columns of data, the left
column represents the old template and the right column represents the
new template. The syntax of the right side is setting values in one of
two arrays, vx and tx. The tx array is used for table elements and the
vx array is used for everything else. The following sample array
provides the upgrade path for three inputs in a section named basic.
array set upgrade_var {
::basic__addr {[set vx(vs_pool__vs_addr) ##]}
::basic__need_snatpool {[set vx(net__snat_type) ##]\
[set vx(net__snatpool) ##]}
::basic__snatpool_members {[set tx(net__snatpool_members) ##]}
}
What this says is that the values from addr input in the basic section
of the old template are moving to the vs_addr input in the vs_pool
section of the new template. The need_snatpool input value is split
into two different inputs. Both the snat_type and snatpool inputs in
the net section will receive the value that came from the
need_snatpool input in the old template. All of the values in the
snatpool_members table in the basic section of the old template are
mapped into the snatpool_members table in the net section of the new
template.
The value ## is a convenience value. It gets translated into the value
stored in the input in the left column. This is the most common value
to use, but another value or input could be substituted. For example,
the following example could be used in the right column.
[set vx(basic__legacy_advanced) [expr { \
(![iapp::is ::basic__snat_1 {No}] || \
![iapp::is ::basic__snat_2 {No}] ) && \
[iapp::is ::server_pools__lb_method_choice default {least-connections-member}] \
?no:yes}]]
This sets the ASO variable basic__legacy_advanced is set not to
whatever value ## would have resolved to, but instead to the result of
the expression. Left side variables require the leading :: while on
the right side, this should be omitted.
Building a Translations Array¶
The translation array defines any changes that need to happen to
choice drop-down inputs. The translation array contains two different
kinds of translations: universal and specific. Universal translations
happen anywhere that value is found. Specific translations only apply
to a particular input.
The following translation array definition contains four universal
translation and specific translations for the input snat in the
section basic.
array set upgrade_trans [subst {
Yes yes
No no
enabled yes
disabled no
basic__snat {
Yes {/#default#}
No {/#do_not_use#}
}
}]
What this array says is that all answers of ‘Yes’ should be changed to
an all lowercase ‘yes’ for any choice element in the template.
Similarly, ‘No’ gets translated to ‘no’, ‘enabled’ gets translated to
‘yes’, and ‘disabled gets translated to ‘no’. The exception to those
rules is that for just the variable basic__snat (the choice element
named snat in the basic section) the value ‘Yes’ is translated to
‘/#default#’ and ‘No’ gets translated to ‘/#do_not_use#’.
This can be made more complicated than just a simple value change. TCL
code can be inserted into this array to make more complicated choices.
The following entry makes a choice during translation.
server_pools__create_new_pool {
{Create New Pool} {/#create_new#}
{Use Pool...} [expr { [info exists ::server_pools__reuse_pool_name] \
? $::server_pools__reuse_pool_name : {/#create_new#} }]
}
This says that for the create_new_pool element in the server_pools
section, translate the string ‘Create New Pool’ to the string
‘/#create_new#’ and if the value is ‘Use Pool…’ then check if the
ASO variable ::server_pools__reuse_pool_name exists. If it does exist
then use its value, otherwise use the string ‘/#create_new#’.
Universal translations are simple string-to-string translations. If
there are spaces in the strings, then they need to be enclosed in
curly braces.
Specific translations are specified with a variable name with no
preceding :: on the left side and a compound value on the right side
containing two columns defining the translations.
Building a Downgrade Array¶
The downgrade array is the simplest of the three. Since most of the
old values are still preserved in the ASO, it is only necessary to map
the tables that need to be mapped back to the old template. The
following sample translation array illustrates its use.
array set downgrade_tbl {
::vs_pool__members server_pools__servers
::net__snatpool_members basic__snatpool_members
}
This says that members table from the vs_pool section of the new
template gets mapped back to the servers table in the server_pools
section of the old template and the snatpool_members input from the
net section of the new template gets mapped to the snatpool_members
input in the basic section of the old template. Leading :: are
required on the left side and should be omitted on the right side.
Converging the Templates¶
Building the upgrade and translation arrays is relatively
straightforward. Making the old and new versions coexist in the same
template and making the active-version auto-detection work is trickier
to describe. There are several things that must happen for this to
work correctly. All of the code for both templates, both presentation
section and implementation section, must exist simultaneously in the
template and there needs to be a way to detect which version to
display. Every input from the old template must still exist in a
section by the same name. You can have new sections that are just for
the new version of the template and new inputs, but you have to keep
all of the old ones, otherwise the upgrade cannot operate.
How separate old and new content needs to be depends on how extensive
the changes are. In some cases one might have two completely different
templates with different section names and no real overlap. In other
cases, there might be some sections that are displayed only for the
older template and some that are displayed only for the newer
template, and some that are shared between them. The shared sections
would be ones that have not changed between versions. On the back end,
the code for both implementation sections needs to be included, with a
check right at the beginning that decides which code is run.
Putting all of the code together into a single file is the easy part.
The more difficult and subtle part detects which version of the
template to display and process. It is necessary to detect from the
APL code if this is a new deployment, a reentrancy into an ASO that
was built using the new-style template, or if the ASO is from the
old-style template. This is accomplished by usurping an input from the
old template and making it perform double duty. This variable is
called the pivot variable. The input in question needs to be a choice
drop-down. It is much easier to handle the logic if it has only two
values. The input needs to be at the top of the top-most section that
has real inputs. If it is at least in the right section, it can be
moved within the section without changing the ASO variable name.
Sections can also be reordered if necessary. Reordering doesn’t change
the ASO variable name.
For example, we use an old template with a first section like the
following, where … represents other lines of inputs that are not
relevant to this explanation.
section basic {
…
…
choice snat default "No" { "No", "Yes" }
…
…
…
}
In this example, we use the snat input. We move it to the top, hide it
from view, change the default, and then add two extra options to it,
so it looks like this:
section basic {
optional ( "HIDE" == "THIS" ) {
choice snat default "no_legacy" {
"Yes",
"No",
"legacy",
"no_legacy"
}
}
…
…
…
}
If this template is entered for the first time, the value will be
‘no_legacy’ because that is the default. However, if this ASO came
from the old template, this value will be either ‘Yes’ or ‘No’. If
this value is either ‘Yes’ or ‘No’, then we know we should be
displaying the old template. In that case, we want to provide the user
with the option to upgrade using the following code:
section basic {
optional ( "HIDE" == "THIS" ) {
choice snat default "no_legacy"
}
# For old-style template
optional ( snat == "Yes"
|| snat == "No" ) {
message deprecated
choice upgrade default "No" display "xxlarge"
…
<Content for old version of this section of the template>
…
}
# For new-style template
optional ( snat == "legacy"
|| snat == "no_legacy" ) {
…
<Content for new version of this section of the template>
…
}
}
If the user selects upgrade and submits, the upgrade process on the
back-end changes the value of basic__snat to legacy so that the state
is preserved. On reentrancy, it will be obvious that the new template
should be displayed, but that this was once an old-style template, so
a downgrade options should be offered.
We have succeeded in detecting in which mode the template should be
displayed and run, but in the process, we have lost access to the
original value that was contained in this field. In order to save this
value, we create a couple of new inputs, one for each possible value
in the old version of the input. This is why we want to use a
drop-down with as few options as possible. In this case, we only need
two. If the original input had five possible values, then we would
need five new inputs. Add something like the following to the part of
this section that only shows up when the template is in legacy,
old-style mode.
optional ( snat == "Yes" ) {
choice snat_1 display "xxlarge" default "Yes" { "No", "Yes" }
}
optional ( snat == "No" ) {
choice snat_2 display "xxlarge" default "No" { "No", "Yes" }
}
We present one optional for each possible value. Inside each, we place
a new version of the old input with its own name. The default for this
version should be the same as the value checked against in the
optional. That way, the one that is displayed will come up with the
right value. On the back-end, you will have to change the code to get
this value from the right input.
Finally, wrap the rest of the section in optional based on the pivot
variable so that old-style sections and new-style sections are only
displayed when that it appropriate.
# For old-style template
optional ( snat == "Yes"
|| snat == "No" ) {
<old-style section>
<old-style section>
<old-style section>
…
}
# For new-style template
optional ( snat == "legacy"
|| snat == "no_legacy" ) {
<new-style section>
<new-style section>
…
}
<shared section>
…
Managing the Pivot Variable¶
There are a few final things needed to make this all work. During
upgrade, we need to change the value of our pivot variable to ‘legacy’
so that the new template pages will be displayed on reentrancy. We
also need to store the original value of the pivot variable somewhere
in the ASO so that we can get it back if the user ever chooses to
downgrade. This is stored in a variable named offload_history. We
accomplish this using the upgrade array. Here is a sample entry in the
upgrade array that handles these two tasks for the pivot variable in
our example.
::basic__snat { \
[set vx(offload_history) ##] \
[set vx(basic__snat) "legacy"] \
}
Appendix A: The Upgrade procedure¶
The upgrade procedure takes two arrays as inputs. The parameter
upgrade_var is the upgrade array described above and upgrade_trans is
the translation array described above. There is no return value.
proc upgrade_template { upgrade_var upgrade_trans } {
upvar $upgrade_var upgrade_var_arr
upvar $upgrade_trans upgrade_trans_arr
# create the new variables from the old
foreach { var } [array names upgrade_var_arr] {
# substitute old variable name for abbreviation "##"
regsub -all {##} $upgrade_var_arr($var) \$$var map_cmd
# run the mapping command from inside the array
if { [catch { subst $map_cmd } err] } {
if { [string first "no such variable" $err] == -1 } {
puts "ERROR $err"
}
}
}
# move variables over and apply translations
set var_mods ""
set var_adds ""
foreach var [array names vx] {
# if the APL variable name is in the translation array,
# then use the custom translation built for that variable.
if { [info exists upgrade_trans_arr($var)] } {
array set sub_arr [subst $upgrade_trans_arr($var)]
if { [info exists sub_arr($vx($var))] } {
set vx($var) $sub_arr($vx($var))
}
array unset sub_arr
# else, if the APL variable value is in the translation array,
# then use the generic translation of that value.
} elseif { [info exists upgrade_trans_arr($vx($var))] } {
set vx($var) [subst $upgrade_trans_arr($vx($var))]
}
# add to tmsh command string
if { [info exists ::$var] } {
append var_mods "\n $var \{ value \"$vx($var)\" \} "
} else {
append var_adds "\n $var \{ value \"$vx($var)\" \} "
}
}
# move tables over
set tbl_mods ""
set tbl_adds ""
foreach tbl [array names tx] {
# convert table from APL format to TMSH format
if { ![llength $tx($tbl)] } {
set tbl_def "column-names none"
} else {
set rows_def ""
foreach apl_row $tx($tbl) {
array set row_arr [join $apl_row]
append rows_def "\n \{ row \{ "
foreach apl_col [array names row_arr] {
append rows_def "$row_arr($apl_col) "
}
append rows_def "\}\}"
}
set tbl_def \
"\n column-names \{ [array names row_arr] \} rows \{ $rows_def \}"
array unset row_arr
}
# add to tmsh command string
if { [info exists ::$tbl] } {
append tbl_mods "\n $tbl \{ $tbl_def \} "
} else {
append tbl_adds "\n $tbl \{ $tbl_def \} "
}
}
# construct the "tmsh modify" command
set cmd "sys application service $tmsh::app_name "
if { [llength $var_mods] } {
append cmd "\nvariables modify { $var_mods }"
}
if { [llength $var_adds] } {
append cmd "\nvariables add { $var_adds }"
}
if { [llength $tbl_mods] } {
append cmd "\ntables modify { $tbl_mods }"
}
if { [llength $tbl_adds] } {
append cmd "\ntables add { $tbl_adds }"
}
# Execute with debug output. This conversion takes place within the
# existing ASO, so tmsh modify is used instead of tmsh create.
debug "TEMPLATE UPGRADE"
iapp::tmsh_modify $cmd
return
}
Appendix B: The Downgrade procedure¶
The downgrade array takes three parameters. The first two, pivot_var
and upgrade_var, are the variable names for the pivot variable and the
upgrade choice. In our example above, those would be basic__snat and
basic__upgrade, respectively. The third parameter is the downgrade
array, described above. There is no return value.
proc ::iapp::downgrade_template { pivot_var upgrade_var downgrade_table } {
upvar $downgrade_table downgrade_tbl_arr
# The ASO variable "offload_history" is used to recover the legacy
# choice a user made about SSL offload. It should be present in all cases.
# This conditional only handles the case where a user has deliberately
# deleted it by manipulating the ASO directly from tmsh.
if { ![info exists ::offload_history] } {
set ::offload_history "No"
}
# BIG-IP erases table contents when the APL optional hides the table.
# Since the prior data is not available, this downgrade must back-convert
# existing table data. Unlike tables, variables remaintact from the
# legacy ASO.
set tbl_def ""
foreach tbl [array names downgrade_tbl_arr] {
append tbl_def "$downgrade_tbl_arr($tbl) \{ "
if { [llength [subst $$tbl]] } {
set rows_def ""
foreach apl_row [subst $$tbl] {
array set row_arr [join $apl_row]
append rows_def "\n \{ row \{ "
foreach apl_col [array names row_arr] {
append rows_def "$row_arr($apl_col) "
}
append rows_def "\}\}"
}
append tbl_def \
"column-names \{ [array names row_arr] \} rows \{ $rows_def \}"
array unset row_arr
} else {
append tbl_def "rows none"
}
append tbl_def " \} "
}
regsub -all {\n} $tbl_def {} tbl_def
set cmd "sys app ser $tmsh::app_name \
variables modify \{ \
$pivot_var \{ value $::offload_history \} \
$upgrade_var \{ value No \} \
\} \
tables modify \{ $tbl_def \}"
debug "TEMPLATE DOWNGRADE"
iapp::tmsh_modify $cmd
return
}
Appendix C: Sample Arrays¶
array set upgrade_var_arr {
::basic__persist_on_source_ip { \
[set vx(offload_history) ##] \
[set vx(basic__persist_on_source_ip) "legacy"] \
[set vx(basic__legacy_advanced) [expr { \
(![iapp::is ::basic__persist_on_source_ip_1 {No}] || \
![iapp::is ::basic__persist_on_source_ip_2 {No}] ) && \
[iapp::is ::server_pools__lb_method_choice default {least-connections-member}] \
?no:yes}]]}
::basic__persist_on_source_ip_1 {[set vx(vs_pool__persistence) ##]}
::basic__persist_on_source_ip_2 {[set vx(vs_pool__persistence) ##]}
::basic__addr {[set vx(vs_pool__vs_addr) ##]}
::basic__port {[set vx(vs_pool__vs_port) ##]}
::basic__protocol {[set vx(vs_pool__protocol) ##]}
::basic__timeout {[set vx(vs_pool__timeout) ##]}
::server_pools__create_new_pool {[set vx(vs_pool__pool_to_use) ##]}
::server_pools__lb_method_choice {[set vx(vs_pool__lb_method) ##]}
::server_pools__create_new_monitor {[set vx(app_health__monitor) ##]}
::server_pools__servers {[set tx(vs_pool__members) ##]}
::server_pools__monitor_type {[set vx(app_health__monitor_type) ##]}
::server_pools__monitor_interval {[set vx(app_health__frequency) ##]}
::server_pools__monitor_tcp_send {[set vx(app_health__tcp_send) ##]}
::server_pools__monitor_tcp_recv {[set vx(app_health__tcp_recv) ##]}
::server_pools__monitor_udp_send {[set vx(app_health__udp_send) ##]}
::server_pools__monitor_udp_recv {[set vx(app_health__udp_recv) ##]}
}
array set upgrade_trans_arr [subst {
{Use Default Profile} /#default#
Yes yes
No no
enabled yes
disabled no
offload_history {
Yes Yes
No No
}
basic__persist_on_source_ip_1 {
Yes {/#default#}
No {/#do_not_use#}
}
basic__persist_on_source_ip_2 {
Yes {/#default#}
No {/#do_not_use#}
}
server_pools__create_new_pool {
{Create New Pool} {/#create_new#}
{Use Pool...} [expr { [info exists ::server_pools__reuse_pool_name] \
? $::server_pools__reuse_pool_name : {/#create_new#} }]
}
server_pools__create_new_monitor {
{Create New Monitor} {/#create_new#}
{Use Monitor...} [expr { [info exists ::server_pools__reuse_monitor_name] \
? $::server_pools__reuse_monitor_name : {/#create_new#} }]
}
}]
array set downgrade_tbl_arr {
::vs_pool__members server_pools__servers
}
The BIG-IP API Reference documentation contains community-contributed content. F5 does not monitor or control community code contributions. We make no guarantees or warranties regarding the available code, and it may contain errors, defects, bugs, inaccuracies, or security vulnerabilities. Your access to and use of any code available in the BIG-IP API reference guides is solely at your own risk.