option_table¶

The Option Table¶

Table Format¶

This table has the following columns:

option_id¶

This column has type integer and is the primary key for this table. Its initial value is zero, and it increments by one for each row.

option_name¶

This column has type text and has a different value for every row; i.e., the names are unique and can act as substitutes for the primary key.

option_value¶

This column has type text and is the value for the corresponding option.

Default¶

The possible values for option_name together with the corresponding default option_value are listed in option_default .

Changing Values¶

This table is special in that you can change almost all its values without invalidating the output tables . There is one exception; see Parent Node below.

Parent Node¶

Unlike other options, if you change the parent node, you must re-run the init_command ; see Changing Values above.

parent_node_id¶

If option_name = parent_node_id the corresponding option_value is an integer that specifies the parent node_id . It is easier (and equivalent) to use parent_node_name instead of parent_node_id ; see below.

parent_node_name¶

If option_name = parent_node_name the corresponding option_value is an string that specifies the parent node_name .

Name or Id¶

If both parent_node_name and parent_node_id are not null, the corresponding node_id and node_name must agree; see node_table .
If parent_node_name is null, parent_node_id is not null and parent_node_id determines the parent node.
If parent_node_id is null, parent_node_name is not null and the node_table is searched to determine the corresponding parent_node_id . this takes some extra time that it is not expected to be significant.

Children¶

The Children corresponding to the parent node.

Data Subset Table¶

Rows in the data table for which the node is not the parent node, or a Descendant of the parent node, are not included in the data_subset_table and the fit_data_subset_table .

Avgint Subset Table¶

Rows in the avgint table for which the node is not a Descendant of the parent node, are not included in the predict_table ; see the heading Note for the predict table .

Zero Sum Constraints¶

zero_sum_child_rate¶

If option_name is zero_sum_child_rate , the corresponding value is a space separated subset of the rate names . For each rate in the list, each age, and each time, the sum of the corresponding child rate effect estimates is constrained to be zero. These are additional constraints in the optimization problem for the fixed effects . The file zsum_child_rate.py contains an example and test using these constraints. Note that for each rate in zero_sum_child_rate , child_nslist_id must be null; i.e., all of the child must use the same smoothing for this rate.

zero_sum_mulcov_group¶

If option_name is zero_sum_mulcov_group , the corresponding value is a space separated subset of the group names . For each group in the list, each age, and each time, and each mulcov_id , the sum of the corresponding subgroup covariate multiplier estimates is constrained to be zero. These are additional constraints in the optimization problem for the fixed effects . The files zsum_mulcov_rate.py and zsum_mulcov_meas.py contain examples and tests using these constraints.

Extra Columns¶

data_extra_columns¶

If option_name is data_extra_columns , the corresponding value is space separated list of extra columns, in the data_table . These columns are not used by dismod_at except that they get displayed in the data.csv file.

avgint_extra_columns¶

If option_name is avgint_extra_columns , the corresponding value is space separated list of extra columns, in the avgint_table . These columns are not used by dismod_at except that they get displayed in the predict.csv file.

Age Average Grid¶

ode_step_size¶

If option_name = ode_step_size , the corresponding option_value is a positive floating point number (greater than zero) that specifies step size used to approximate the solution of the Ordinary Differential Equation . It is also the step size is also used to approximate the integrals in the definition of the average integrands . The default value for ode_step_size is 10.0 which is reasonable if all the rates iota , rho , chi , and omega are all less than 0.1 .

age_avg_split¶

If option_name = age_avg_split , the corresponding option_value is a space separated set of age values at which to split the age integration. This split is both for the computation of the average integrands and for the solution of the ODE; see Age Average Grid . If this value is null, age_avg_split is the empty set.

rate_case¶

This option’s name should be changed to ode_method.

no_ode¶

The rate_case value does not matter for the following integrands: Sincidence , remission , mtexcess , mtother , mtwith , relrisk , mulcov_ mulcov_id . If all of your integrands are in the set above, you can use no_ode as the rate case and avoid having to worry about constraining certain rates to be positive or zero.

trapezoidal¶

If option_name = rate_case and option_value = trapezoidal , a trapezoidal method is used to approximation the ODE solution.

iota_zero_rho_zero¶

If option_name = rate_case and option_value = iota_zero_rho_zero , the parent smoothing for iota and rho must always have lower and upper limit zero. In this case an eigen vector method is used to approximate the ODE solution.

iota_pos_rho_zero¶

If option_name = rate_case and option_value = iota_pos_rho_zero , the parent smoothing for iota must always have lower limit greater than zero and for rho lower and upper limit zero. In this case an eigen vector method is used to approximate the ODE solution.

iota_zero_rho_pos¶

If option_name = rate_case and option_value = iota_zero_rho_pos , the parent smoothing for rho must always have lower limit greater than zero and for iota lower and upper limit zero. In this case an eigen vector method is used to approximate the ODE solution.

iota_pos_rho_pos¶

If option_name = rate_case and option_value = iota_zero_rho_pos , the parent smoothing for iota and rho must always have lower limit greater than zero. In this case an eigen vector method is used to approximate the ODE solution.

Optimize Fixed and Random¶

The following options control the Ipopt optimization of both the fixed and random effects:

derivative_test¶

If option_name is derivative_test_fixed (derivative_test_random ), the corresponding option_value is one of the choices below. The default value for this option is none . (Under normal circumstances, you should use none because the other choices will take more execution time and are for program validation.)

option_value	Description	Restrictions
`none`	do not perform derivative test	none
`first-order`	test first derivatives	none
`second-order`	test first and second derivatives	in fixed case quasi_fixed must be false.
`only-second-order`	only test second derivatives	in fixed case quasi_fixed must be false.
`adaptive`	a step size adaptive first order derivative test	only for fixed case
`trace-adaptive`	trace adaptive test on standard output	only for fixed case

max_num_iter¶

If option_name is max_num_iter_fixed (max_num_iter_random ), the corresponding option_value is an integer greater than or equal -1 that specifies the maximum number of iterations. This is called max_iter in the Ipopt documentation. If Ipopt is run with zero iterations, the solution in the fit_var_value column of the fit table may not correspond to the starting fixed effects; i.e., the fixed effects in the start_var_table table; see the heading bound_frac_fixed below. In the special case where max_num_iter_fixed is -1 , the solution in the fit table is the start_var table value for the fixed effects and the corresponding optimal value for the random effects.

print_level¶

If option_name is print_level_fixed (print_level_random ), the corresponding option_value is a positive integer between 0 and 12 inclusive that specifies the print level for optimizing the fixed effects (random effects). Zero, corresponds to no printing and is the default for dismod_at (the value 5 is the normal default for Ipopt). At least one these two print values (for the fixed and random effects) should be zero.

accept_after_max_steps¶

If option_name is accept_after_max_steps_fixed (accept_after_max_steps_random ), the corresponding option_value is a positive integer specifying the maximum number of backtracking steps to take before accepting a line search point.

tolerance¶

If option_name is tolerance_fixed (tolerance_random ), the corresponding option_value is a positive floating point number that specifies the desired relative convergence tolerance for optimizing the fixed effects (random effects). (This is called tol in Ipopt documentation.) For the fixed effects, this tolerance is relative to the derivatives of the fixed effects objective at the value of the fixed effects in scale_var_table .

Optimize Fixed Only¶

The following options control the Ipopt optimization of the fixed effects:

quasi_fixed¶

If option_name is quasi_fixed , the corresponding possible values are true or false . If it is true , a quasi-Newton method is used to optimize the fixed effects. Otherwise a full Newton method is used.

bound_frac_fixed¶

If option_name is bound_frac_fixed the corresponding option_value is greater than zero and lese than or equal one half. It determines the maximum an initial value in the start_var_table , is moved to be interior to the corresponding upper and lower bounds. If there is an upper or lower bound for a fixed effect, then both are present in the prior_table and

bound_frac_fixed * ( upper - lower )

is the maximum that start_var table value is moved. This is called bound_frac in the Ipopt documentation and \(\kappa_2\) in the corresponding paper. The default value for this option is 1e-2 . Note that bound_push in the Ipopt documentation (\(\kappa_1\) in the paper) is set to be effectively zero.

limited_memory_max_history_fixed¶

if option_name is limited_memory_max_history_fixed , the corresponding option_value is the number of most recent iterations that are taken into account for the limited-memory quasi-Newton approximation.

Optimize Random Only¶

The following options control the Ipopt optimization of the random effects:

method_random¶

If option_name is method_random , option_value is either ipopt_solve or ipopt_random . The method ipopt_solve is part of the standard CppAD package. The method ipopt_random is a special purpose interface to ipopt designed to optimizer the random effects for the cppad_mixed package (which should be faster).

bound_random¶

If option_name is bound_random , option_value is a bound for the absolute value of random effects . If bound_random is null , plus and minus infinity is used for the bounds; see Bounds . This bound does not apply for random effects that have their upper and lower limits equal; see constant child value priors . In addition, if a child node does not have any data, its rate random effects automatically use a random bound of zero.

The theory for cppad_mixed does not include bounds for the random effects.
The bound bound_random does apply to random effects corresponding to constant child value priors . The theory for fitting with constant random effects is just fine because they are treated a parameters, and not random effects, in the Laplace approximation.
The special case where bound_random is zero, fits as if there were no random effects except for the fact that the resulting optimal value can be used as a starting point for fitting with random effects; see set start_var fit_var and fit_fixed_both.py .
Fitting with no random effects corresponds to a null value for all the child_smooth_id and child_nslist_id in the rate table. This is different from bound_random zero which includes the random effects (with value zero) in the model_variables .
Note that the theory used to compute asymptotic samples of the fixed effects, does not take bound_random into account. On the other hand; the samples of the random effects are within the limits specified by bound_random .

Other Database¶

other_database¶

If option_name is other_database , the corresponding option_value is the name of a database where some of the input tables are specified. If this is a relative path, it is now relative to the directory where the directory where Primary Database is located.

other_input_table¶

If option_name is other_input_table , the corresponding option_value is a space separated list of input table names that are in other_database . This is useful when the same table used by many databases. The option table name cannot be in this list.

meas_noise_effect¶

If option_name = meas_noise_effect , option_value determines how the average noise effect changes the measurement noise. The possible option_value are: add_std_scale_all , add_std_scale_none , add_var_scale_all , add_var_scale_none .

warn_on_stderr¶

If option_name is warn_on_stderr , the corresponding possible values are true or false . If the value is true, warning messages are written to standard error. In either case, these messages are recorded in the log_table . Error messages are always written to standard error and recorded in the log table.

minimum_meas_cv¶

This option is no longer available because it is now integrand specific; see minimum_meas_cv .

hold_out_integrand¶

If option_name = hold_out_integrand , the corresponding option_value is a space separated list of integrand_names . (The integrand names cannot begin with mulcov_ .) All the data for the corresponding integrands is held out during any fit. See the fit command hold_out documentation.

random_seed¶

This is a non-negative integer used to seed the random number generator that is used by dismod_at . If this value is zero, the clock is used to seed the random number generator; see random_seed in the log table.

compress_interval¶

If option_name = compress_interval , the corresponding option_value is a space separated list with two elements. The first element is called the age_size and the second element is called time_size . If age_upper is equal to age_lower and time_upper is equal to time_lower , the model for the data does not require integration over an interval. The average for data table age intervals with

age_upper - age_lower <= age_size

is approximated by the value at age

( age_upper + age_lower ) / 2.0

The average for data table time intervals with

time_upper - time_lower <= time_size

is approximated by the value at time

( time_upper + time_lower ) / 2.0

The default value for age_size and time_size is zero; i.e., no age or time compression.

trace_init_fit_model¶

If option_name is trace_init_fit_model , the corresponding possible values are true or false . If it is true , a trace of the initialization of the fit_model data structure is printed on standard output. This gives one an indication of progress for large problems where this initialization takes a long time.

splitting_covariate¶

If option_name is splitting_covariate , the corresponding value is the name of the splitting covariate. The covariate values in rate_eff_cov_table can be different for each value of the splitting covariate. For example, the covariate values might be different for each value of sex. If the splitting_covariate is not null , the rate_eff_cov table must be non-empty.

asymptotic_rcond_lower¶

If option_name is asymptotic_rcond_lower , the corresponding value is a lower bound for the reciprocal condition number of the Hessian of the fixed effects objective. This Hessian is used as an approximation for the information matrix which sampling using the asymptotic method. The reciprocal condition is always between zero and one. If the actual condition number is less than asymptotic_rcond_lower, the asymptotic sample method will fail. If print_level for optimizing the fixed effects is non-zero, the actual reciprocal condition number is printed after asymptotic sampling of the fixed effects.

Example¶

The files option_table.py contains an example option table.