Question about hierarchical Q-Learning

[matouk98]

Hi, hanjun. Thanks a lot for your great work! I have a question about the hierarchical Q-Learning mentioned in the paper. In equation 11, there are 2M Q functions and the paper claims only two distinct parametrizations are enough. However, when I read the code /node_attack/q_net_node.py, in line 163, there initialize num_steps Q networks which are different from the paper. Does my understanding have any mistakes? Looking forward to your reply!!

[Hanjun-Dai]

Hi there,

Thanks for your interest in our work. As in our paper we only add/delete one edge, so it is equivalent.

[cxw-droid]

Hi,

I can see adding one edge in the code

graph_adversarial_attack/code/graph_attack/rl_common.py

Line 77 in f2aaad7

g.add_edge(self.first_nodes[i], actions[i])

But I have not found where an edge is removed. Or the code did not implement edge removing? Thanks!

[Hanjun-Dai]

Hi, you can check the folder for node attack case.

[cxw-droid]

Thank you for your reply. For node attack, I only find adding edges:

graph_adversarial_attack/code/node_attack/node_attack_common.py

Line 33 in f2aaad7

def add_edge(self, x, y, z):