Multi-Agent Reinforcement Learning for Intrusion Detection: A case study and evaluation

I gave this seminar on Feb 22nd.
Artificial Intelligence Group. Computer Sciences, University of York

Abstract:

In this seminar I will present an architecture of distributed sensor and decision agents that learn how to identify normal and abnormal states of the network using Reinforcement Learning (RL). Sensor agents extract network state information using tile-coding as a function approximation technique and send communication signals in the form of actions to decision agents. These in turn generate actions in the form of alarms to the network operator. By means of an on-line process, sensor and decision agents learn the semantics of the communication actions without any previous knowledge. In this presentation I will describe the learning process, the operation of the agent architecture and the evaluation results of our research work.

The presentation is here:




And a video of a Denial of Service Attack. Disclaimer: It may be disturbing for certain audience (it contains cheesy music from ABBA)

Gnuwin32

This is an alternative to have some GNU applications (e.g. grep, gawk, ls, wget, etc.) in your windows machine without installing cygwin. It is not so powerful as Cygwin but it does the trick to install some applications to run some scripts. For me it has been useful to run some scripts with gawk and wget.
To get it, download from http://getgnuwin32.sourceforge.net/ It will probably redirect you to the download page of sourceforge. I will suggest downloading and installing all the packages instead of installing just the application that you need (e.g. wget). The package manager will then take care of any updates that your applications will need in the future.
After download and run the exec file, it will prompt asking for a destination to decompress the file. You can choose whatever you want. You will need to move it to someplace else later, so a good place may be “My Documents”. To install, follow the instructions. This is just a summary; if you want details you can check the readme file in the package.
1) Configure wget if you are behind a proxy. (use bin\wget.ini)
2) Edit and select your mirror in download.bat
3) If 1 and 2 worked, it will start to download stuff (all the applications pacakes)
4) After finishing the dowload run install.bat. It will start to decompress the packages
5) The following is optional, but I suggest to do it because it will be easy to work with the programs in gnuwin32
6) You will see now a folder “gnuwin32”. Now you can move the entire directory to “C:\Program Files”
7) After moving the entire gnuwin32 run “update-links.bat” to update any orphan link.
8) Copy the folder “Star Menu” inside gnuwin32 to the Start Menu of your windows desktop. When executed the shortcuts inside will automatically start a cmd in the path of gnuwin32.
9) That’s all. Enjoy.

There are some more optional things that you can do. Because I do not use them and they can “mix up” some original windows applications with the same name in the gnu package I will not explain them here. If you want to do it, check the readme file.

Packages in GNUWin32

A worm that strikes back

The last August 9th the REN-ISAC from the University of Indiana warned the academic community about the Storm Worm infected machines. After scanned, machines that are infected strike back with a flood DoS attack to the source of the scanning. The process seems to be automated according to the note.

Although the warning was issued to universities in U.S. I am sure that it will also affect to other universities and enterprises that have the scanning of hosts as one of their security policies.

More notes:
Information Week
The Register

Spock or Spooky

This has nothing to do with my research, but any way it was a little bit amusing and worrisome to do some research about this topic. Few weeks ago I knew about Spock, a site for searching people. I was eager to jump in and to test what it was about (I did something similar for LinkedIn, Facebook, Myspace, etc. sometime ago) but then I thought. Even that someone offered me an invitation I stopped and I wonder. Do I really want all my personal data to be in just one place?

I mean, my data is there around my blogs, my website, my profile in I do not how many places. You just need to do some Google research to find my contacts details and some information about me. So, is there any difference between “google” me or search about me in Spock? Well, there is. I am not in Spock. Nice, isn’t it?

Well, but someday for sure I will, so, there will be any difference then? I think yes, while searching about people with Yahoo, Google or any other search engine you have to go around several pages to get all the data, while in Spock, you get it with just one or two clicks (depending how common is the name you are looking for). The implications are so great (I am been sarcastic if you haven’t noticed it), you can have all the need to make some online frauds, crack passwords, stole identities, etc. The possibilities are unlimited. I am being paranoiac, yeah, may be. In the other side, may be hackers will not use it any way, today is a little bit slow and online scammers rely in better applications than Spock to profile people (just read this). So, in the end I think that it will be very helpful to track some of your old friends, colleagues and classmates. And why not, to amuse you a little bit finding curious details of people that share the same name than your friends or best, to know details that you did not know about your friends. Just to mention I learn Zodiac Signs, weird hobbies, sexual interests, trips, past relations, etc. Every bit of information that is there but they did not share with you, you just need to dig a little to find it.
Until today, if you are trying to find some friends to get in touch, better use other methods such as Google. The database of Spock is still very small. Finally, the concept is not new. There are some other sites that do the same. The thing with Spock is that the marketing played and important role to bring it to the spot light.

Digg and Microsoft

This is not about my research but I found interesting to comment about it.

Few days ago Digg and Microsoft signed an agreement where Microsoft will be the provider of contextual advertising.

I predict some change in Digg during the next six months:

- You won't be able to criticize Microsoft, if you do that, then you will be banned
- You will need to use IE with DRM
- If you comment about Linux, you could be sued by infringement of bogus MS pattens
- You could get infected by a worm just to click in posts
- You won't be allowed to submit news about Linux, Apple or Google
- The site sometimes will display a blue screen

True Random Generator Service

  It is based on the " 'Quantum Random Bit Generator' (QRBG121), which is a fast non-deterministic random bit (number) generator whose randomness relies on intrinsic randomness of the quantum physical process of photonic emission in semiconductors and subsequent detection by photoelectric effect".

  Why is this good? well because software cannot generate real random numbers, they just can generate pseudo random numbers.

 

  If interested there is a QRBG Service that you can access it online. Today there are C++ libraries, MathLab toolbox and CLI access. They are also planning to support web access. 

Spammers using DDoS attacks

I read in a post in the SANS website that ansti-spam groups websites are under a DDoS attack. It is interesting the point of view of the post's author about seeing this as a desperate action from the spammers groups.