Questions about function test_after_training()

[wisdomGEsLA]

Sorry to be a bother!
When I try to apply my own model to this code, I find that there may be no test using test data after traing if the length of parameter list is not 1, i.e no calling test_after_training() .
So if I want to conduct a test after training under each parameter combination, should I add an test_after_training() after main like this?

def main_IPS():
    # IPS_lambda_list = [10, 15, 20, 25, 30, 35, 40, 45, 50]
    IPS_lambda_list = [opt.IPS_lambda]
    if len(IPS_lambda_list) == 1:
        opt.show_performance = True
        main()
        if not opt.test_only:
            test_after_training()
    else:
        opt.show_performance = False
        for opt.IPS_lambda in IPS_lambda_list:
            print('\nIPS_lambda = %d' % opt.IPS_lambda)
            with open(opt.log_path, 'a+') as f_log:
                f_log.write('\nIPS_lambda = %d\t' % opt.IPS_lambda)
            main()
            test_after_training()

or maybe I'm just misunderstanding the code?

[zzhUSTC2016]

Hello, thanks for your interest!
If you have trained a model, you can set opt.test_only as True, the code will run

if opt.test_only:
model.load_state_dict(torch.load(opt.model_path_to_load))
if opt.method == 'TIDE' and opt.backbone == 'MF':
torch.save(model.q, opt.q_path)
evaluator.run_test(0)
return evaluator.best_perf

If you want to search for the best parameter, the design here is getting performance on validation set and pick the best one, then evaluating it on test set to get the final performance. It's not recommended to conduct evaluation on test set under each parameter combination. (That may cause unfair experiment result)

Your understanding is right. If you want to do so, you can monitor the performance on test set like this.

[wisdomGEsLA]

Hello, thanks for your interest! If you have trained a model, you can set opt.test_only as True, the code will run

if opt.test_only:
model.load_state_dict(torch.load(opt.model_path_to_load))
if opt.method == 'TIDE' and opt.backbone == 'MF':
torch.save(model.q, opt.q_path)
evaluator.run_test(0)
return evaluator.best_perf

If you want to search for the best parameter, the design here is getting performance on validation set and pick the best one, then evaluating it on test set to get the final performance. It's not recommended to conduct evaluation on test set under each parameter combination. (That may cause unfair experiment result)

Your understanding is right. If you want to do so, you can monitor the performance on test set like this.

Sorry for the late reply! Your answer really helps!
Recently I want to run TIDE on other datasets, after reading the cppcode directory, I think maybe I just need to change the file name in cpp codes?
For exmaple like this
if ((fp = fopen("./data/Amazon-CDs_and_Vinyl/item_interactions.csv", "r")) != NULL)
then maybe we can run the code on Amazon-Beuaty by change it like this, of course when We have had the item_interactions.csv for Amazon-Beuaty
if ((fp = fopen("./data/Amazon-Beuaty/item_interactions.csv", "r")) != NULL