optimisation has probably not converged to the maximum likelihood-hessian is not positive definite

[yinglittxs]

Hi, professor Chris Jackson. When running the below code, I encountered a problem. I have tried the methods mentioned in the manual, but it didn't work. Could you please take a look and see if there is a problem? Is it because my subgroup cannot handle so much covariate information? But these covariates are indeed necessary for adjustment, I am very troubled. If you could help me, I would be extremely grateful.

fit<-msm(fomular=dmms~time_year,
subject=id,
data=combine_prsfhbmi_group4
qmatrix=Q
covariates=factor(BMI_group)+age_scale+factor(sex)+factor(cohorts)+factor(area)+factor(region)+factor(CSMOKE)+factor(drink),
obstype=1,
hessian=T,
method="BFGS",
control=list(fnscale=500000,maxit=10000),
death=3
)

warning message: optimisation has probably not converged to the maximum likelihood-hessian is not positive definite

[chjackson]

I can't tell from this, but the commonest cause of bad convergence in msm is either:

• not enough data on particular transition rates and how these vary with covariates

• a qmatrix with a structure that allows too many transitions in continuous time. This should be chosen based on what is scientifically realistic in continuous time, not on what transitions over an interval are observed. See the course notes.

or both of these. You have a lot of covariate effect parameters in your model. I'd advise simplifying this model, as the manual advises. Start with a small model, and then build it up according to what the data supports. You might assume particular covariates only affect particular transitions. Or constrain some effects to be equal to each other.

[yinglittxs]

Dear Professor Chris Jackson, Thank you for your advise. Maybe the model should be simplified. I would like to ask, is there any problem with my current code?

[chjackson]

I cannot see any obvious errors in the code (apart from typos that look like the result of copying by hand: fomular and no comma after Q), but I'm unable to tell whether this is an appropriate model for your data.

[yinglittxs]

Thank you very much~

[yinglittxs]

I noticed a peculiar phenomenon. Sometimes, when the number of observations and transfers in this set of data is large, this error is frequently reported. However, when the data volume is small, no error is reported. At first, I thought it might be due to insufficient data to support it, but it seems that it's not entirely so.

[chjackson]

Look at your estimates and confidence intervals, as well as whether or not there is an error message. Commonly in situations like this, the confidence intervals are implausibly wide, which would suggest the likelihood is flat for that particular parameter, that is, there is no information in the data about it.

[yinglittxs]

Thank you very much~