Friday, 25 April 2008

NS-2 Memory exhaustion

1078404057

I recently started to run a big simulation (68 agents and nodes) in NS-2 to test my intrusion detection algorithm using reinforcement learning. When I ran the simulation for more than x time, it started freezing the host and it ended killing the process. I look for errors in the code and nothing. I freed some disk space thinking that it could be that the log files were using all the available space. It worked a little until I ran the configuration x + y time.


I suspect about memory use and I increase the memory in the host machine (I was using vmware, so it was easy) with good results. However, as I increased the simulation time the solution became and endless cycle (that will end in no more memory available in the machine hosting the vmware). I started looking for problems in how I was using the memory in my code. I found some links about how to debug memory allocations in NS. I must say that I could not make them work, any way the links are here:


NS-2 debugging tips
dmalloc


Of course I sent an e-mail to the ns-users e-mail list, and as always it was useless (it seems that anybody likes to answer smart questions and newbies always post dumb ones -that no body replies either -). After reading the ns-manual again, I found that I could (or I must I am not sure) free the packets that I used. It is that I developed a new type of agent. The interaction and information shared between agents is of course through special packets that I define.


So, the call is:


Packet::free(pkt);

I call it in the method that receives the packet just after reading the packet data that I need:

void RL_MAgent::recv(Packet* pkt, Handler*)
{
// Access the IP packet
hdr_ip *iph = hdr_ip::access(pkt);
// Access the RL header for the received packet:
hdr_rl* hdr = hdr_rl::access(pkt);
double stime = hdr->send_time_;
int ptype_ = hdr->p_type_;
int nodeid_ = hdr->node_id_;
int src = iph->saddr();
int dest = iph->daddr();
int srcport = iph->sport();
float now_ = Scheduler::instance().clock();
Packet::free(pkt);

if (ptype_ == T_START)
{ ...


As result, my simulation only needs around of a steady 10MB of memory to run.

2 comments:

Giovane said...

Hi Arturo,

Very nice post about this issue with ns-2.
I'm having the same problem at this moment -- ns2 is consuming 1.7GB of RAM and it's running pretty slow.

Following your suggestions, I decided to check the code of my TCP sink receiver agent (set sink [new Agent/TCPSink/Sack1). And that line (Packet:free(pkt)) is there, just after sending the ACK back.

I will have to investigate more -- but at least this post brought some guidelines.

thanks

Giovane

Arturo Servin said...

Good it was useful. Good luck to find the culprit.