\(\newcommand{\B}[1]{ {\bf #1} }\) \(\newcommand{\R}[1]{ {\rm #1} }\) \(\newcommand{\W}[1]{ \; #1 \; }\)
fit_command#
View page sourceThe Fit Command#
Syntax#
dismod_at
database fit
variablesdismod_at
database fit
variables simulate_indexdismod_at
database fit
variables warm_start
dismod_at
database fit
variables simulate_index warm_start
database#
Is an
http://www.sqlite.org/sqlite/ database containing the
dismod_at
input tables which are not modified.
variables#
This argument is fixed
, random
or both
.
If it is fixed
(random
)
only the fixed effects (random effects) are optimized.
If it is both
, both the fixed and random effects
are optimized.
fixed#
This option optimizes the fixed effects with the
random effects corresponding to
constant child value priors
at their constrained values and the other random effects
constrained to be zero.
(This is equivalent to fitting with
bound_random equal to zero.)
This is useful when one uses fitting with no random effects as
a starting point for fitting with random effects; see
set_command ,
start_var_table , and
fit_fixed_both.py .
This enables one to see the different between the two
fits in the log_table
(as apposed to changing
bound_random
in the option
table).
random#
This optimizes the random effects with the fixed effects set to their starting values; see start_var_table .
both#
simulate_index#
If simulate_index is present, it must be less than number_simulate .
data_sim_table#
If simulate_index is present, this is an extra input table. The data_sim_value entries, corresponding to simulate_index , are used in place of the data table meas_value entries. All the rest of the data table values are the same as when simulated_index is not present; e.g., meas_std comes from the data table.
prior_sim_table#
If simulate_index is present, this is an extra input table. The prior_sim_value entries, corresponding to simulate_index , are used in place of the prior table mean entries. All the rest of the prior table values are the same as when simulated_index is not present; e.g., std comes from the prior table.
warm_start#
If warm_start
is at the end of the command, the
ipopt_info_table
written by the previous fit,
is used to start the optimization of the fixed effects
where the previous fit left off.
This is intended to be used in the following cases:
Termination of previous fit is due to reaching max_num_iter_fixed .
The tolerance for the fixed or random effects been changed.
Other options besides those listed above, should be the same as for the previous fit.
data_subset_table#
Only the data table rows with data_id that also appear in the data_subset table are included in the fit and residuals.
hold_out#
A data table meas_value is held out from the fit, but included in the residuals, if any of the following conditions hold:
The data table hold_out is non-zero.
The corresponding data_subset table hold_out is non-zero; see hold_out_command .
The data table integrand corresponds to an integrand_name that is in the option table hold_out_integrand list
After the optimal variable values are found, the hold out residuals are computed so one can check the predictive validity for hold out data. The residuals for hold out data (other data) are computed once (many times) for each fit.
Output Tables#
fit_var_table#
A new fit_var_table is created each time this command is run. It contains the results of the fit in its fit_var_value column.
fit_data_subset_table#
A new fit_data_subset_table is created each time this command is run. It is a comparison of the model and data corresponding to the fit results.
trace_fixed#
A new trace_fixed_table is created each time a
fit fixed
or fit both
command is run.
It contains a trace of the corresponding optimization.
hes_random_table#
A new hes_random_table is created each time this command is run
with variables equal to random
or both
.
The Hessian of the random effects objective is written in this table.
If simulate_index is present (is not present) the Hessian corresponds
to the simulated measurements in the data_sim_table
(measurements in the data_table ).
mixed_info#
A new mixed_info_table table is created each time this command is run.
ipopt_info_table#
The fixed effect are optimized when
variables is equal to both
or fixed
.
In the case a new ipopt_info
table, corresponding to the final
fit for the fixed effects, is written to the ipopt_info table.
The contents of this table are unspecified; i.e., not part of the
dismod_at API and my change.
Random Effects#
A model has random effects if one of the
child_smooth_id or
child_nslist_id is not null
.
In some cases it helps to
first fit with variables equal to fixed
and then fit with variables equal to both
.
Convergence Problems#
The derivative of the fixed effects objective, at the prior mean for the fixed effects, is used to scale the fixed effects objective. If the optimal value for a fixed effect is equal to its prior mean, its partial derivative at the prior mean is often near zero. Near zero derivatives, at the prior mean, cause the scaling to be unstable and result in convergence problems. You may be able to fix this by using a uniform distribution, and a different prior mean, for a fixed effect that does not change. If this does not work, the data will not determine that fixed effect and you may have to constrain it to have the value you want.
Ipopt Options#
Some of the options in the option_table are for controlling
Ipopt.
You can override these options, or set other options, using a file called
ipopt.opt
in the directory where a fit command is run.
Doing so not supported because dismod_at
expects certain options to be set a certain way.
The set of such options that is not specified and may change with time.
Example#
The file fit_command.py contains an example and test using this command.