Pygame Subset for Android - Soft Keyboard

The guys over at pygame.renpy.org have out-done themsleves with their work on the pygame subset for android development. It can be a bit tricky at first if you're not familiar with ant builds or android development, but for the most part they've made it pretty simple to pick up. If I can figure it out, anyone can do it. So first let me just say thanks to all who have worked on the project.

That being said, there are a number of things I wish worked better. Since the pygame mixer wasn't included, the subset allows you to take advantage of the android mixer. Unfortunately the android mixer isn't quite as full-featured in it's implementation here. The soft-touch keyboard is in the same boat. You can import the android subset, and even have the app call the keyboard, but to my knowledge (and this may be completely wrong) you can't actually watch for these key events. I've tried a multitude of methods and searched for hours on how to correctly do this with the pygame subset for android. I've found absolutely nothing that can show me how to correctly implement this.

So what I ended up doing was writing my own soft-keyboard function. I've tested it and it works great with android apps & games.

Here's basically how it works:

def makeKey(x, y, letter):
   global keyArray
   temp = pygame.image.load('key.png')
   if letter == 'enter':
     temp = pygame.image.load('longkey.png')
   limage = pygame.transform.scale2x(temp)
   lrect = limage.get_rect()
   lrect.topleft = [x,y]
   keyArray.append((limage,lrect,letter))
#map out whole keyboard using makeKey()
def makeKeyboard(x,y):
   letters = ['q','w','e','r','t','y','u','i','o','p','a','s','d','f','g','h','j','k','l','z','x','c','v','b','n','m','enter']
   x_move = 0
   y_move = 0
   for i in letters:
     if i == 'a':
       y_move = 40
       x_move = 0
     if i == 'z':
       y_move = 80
       x_move = 0
     makeKey(x+x_move, y+y_move, i)
     x_move+=40
#draw keyboard and write in letters
def drawKeyboard(color,font,surface):
   global keyArray
   for i in keyArray:
     surface.blit(i[0],i[1])
     drawText(i[2],color,font,surface,i[1].center[0],i[1].center[1],True)
#used for drawing text onto the key surface
def drawText(text, color, font, surface, x, y, center):
   textobj = font.render(text, 1, color)
   textrect = textobj.get_rect()
   textrect.topleft = (x, y)
   if center:
     textrect.center = (x, y)
   surface.blit(textobj, textrect)

Then you can do something like this to implement it:

def main():
   #keyboard start coord
   startKey_x = screen.get_size()[0]/2
   startKey_y = screen.get_size()[1]/2

   #create images and rects for keyboard
   makeKeyboard(startKey_x,startKey_y)

   #temp string for testing
   clickedLetters = ""

   clock = pygame.time.Clock()

   while 1:
     clock.tick(60)
     global MOUSEDOWN
     global keyArray

     for event in pygame.event.get():
       if android:
         if android.check_pause():
           android.wait_for_resume()
       if event.type == pygame.QUIT:
         terminate()
       if event.type==pygame.MOUSEBUTTONDOWN:
         MOUSEDOWN=True
         for i in keyArray:
           if i[1].collidepoint(event.pos):
             if i[2] == 'enter':
               pass
             else:
               clickedLetters+=i[2]
       if event.type==pygame.MOUSEBUTTONUP:
         MOUSEDOWN=False
     drawKeyboard(black,font,screen)
     drawText(clickedLetters, black, font, screen, (800/2), (480/4), True)
     pygame.display.flip()

here's an example implementation

So right before the game loop, you create the keyboard. The makeKeyBoard function just needs a starting position and it will create each key and give it it's own rectangle, image and coordinate.

Inside the main loop, under the event listener, you have it watch for mousebuttondown events. When the app is run on android, this equates to a screen press. It uses the event position to determine if there was a collision with a keyboard rectangle. You can map these collisions to any functions you want. You'll notice my keyboard only has letters and the enter key, but you could set up a full keyboard in much the same way.

This likely isn't the best way to implement a soft-keyboard, but it works which is more than I can say for the android soft-touch keyboard using the pygame subset. If anyone else has a better way to do this, or they managed to figure out how to implement the android soft-touch keyboard, I'd love to hear from you. Especially if you figured out the android soft-touch. I'd love to see how you get the events into python.

-Newt

Moar Pygame Sprite Movement

There are a number of ways to have python move game objects around the screen. Up until recently I hadn't really tried much other than tying keys to a function that would move a rectangle object by an offset.

Something like this:

*** in the sprite class***
def change_pos(self, x, y):
  self.change_x += x
  self.change_y += y

*** in the main loop ***
for event in pygame.event.get():
  if event.type == KEYDOWN:
    if event.key == K_UP:
      player.change_pos(0, -PS)

I'd set up a global for player speed (PS) that I could hand to the change_pos function as a positive or negative value based on the direction the player wanted to go.

This way works fine, but it's not really all that flexible. What if you wanted to have the player sprite keep moving a bit after the player releases the movement keys? For that you'd need to use a bit of physics. There are a ton of different ways to do something like this, and I've read through some tutorials on it. Since I'm not very good with math though, most of it was way over my head. I came up with a very simplistic approach to the whole thing.

I created a sprite object with an attribute to store a list of forces that will act as inertia to be applied to the object. Then I took out the change_pos() function and replaced it with this:

def apply_force():
  for i in self.forces:
    i -= friction
  if self.up:
    self.forces[0] += speed
  if self.down:
    self.forces[1] +=speed
  if self.left:
    self.forces[2] +=speed
  if self.right:
    self.forces[3] +=speed

In the player sprite's update function, I can say:

def update():
  self.apply_force()
  self.rect.top += (self.forces[1] - self.forces[0])
  self.rect.left += (self.forces[3] - self.forces[2])

You can play around with the values that you use for speed, and friction. Speed is how fast your player object gains speed and friction is how fast the object is slowed. You may want to add something to the end of your apply_force() function to cap the forces. I used something like this:

for i in self.forces:
  if i > 200:
    i=200
  if i < 0:
    i=0

Also, if you want, you can have your main loop use actual time to update instead of FPS which can be affected by the power of the system you're running on. Up until recently I had been using the following in my main loop:

clock = pygame.time.Clock()
while 1:
  clock.tick(60)

It actually makes more sense to use the following:
FPS = 60
clock = pygame.time.Clock()
while 1:
  milliseconds = clock.tick(FPS)
  seconds = milliseconds / 1000.0

Then change the player class update function to take seconds as an argument. This will give you a much more consistent movement:


  self.rect.top += (self.forces[1] - self.forces[0]) * seconds
  self.rect.left += (self.forces[3] - self.forces[2]) * seconds

There are of course, more accurate ways to produce the same effect, but I found this pretty simple to understand and easy to implement.

-Newt

ArcSight Export Parser Script

For those of you not familiar with it, ArcSight is a pretty robust correlation engine. I've been working with it for quite a while now, and I've never really been happy with the case management system they have. When an alert is triggered, it pulls in some information and saves it to a Case. Unfortunately the management of these cases are kind of messy and can easily become unwieldy for large environments. If you have HP Openview, you can have ArcSight export the cases directly to your ticketing system. But what if you use another ticketing system or a custom built ticketing system? Well, then you can export the files to the system and write your own API to parse the export files and send to the ticketing system of your choice.

That's what I've been working on the last few weeks. With an ArcSight ESM appliance, I'm limited to using bash, perl (limited libraries), or python 2.4. Since I'm more familiar with python, I went that route. Although, you could easily write a parser in perl. In fact, ArcSight has an approved perl script that parses these exports and formats them to a customized email, so you can route events that way. Still python was what I went with.

It's still a work in progress, but here's the basic functions:

# simple function to consolidate headers from base events
def getHeaders(allevents):
   x = []
   for i in range(len(allevents)):
     for thing in allevents[i]:
       if x.count(thing[0]) == 0:
         x.insert(i,thing[0])
   x.reverse()
   return x

# creates an HTML table for use in our web-based ticketing system
def createHtml(baseEventList):
   if os.path.exists('tmp_alert/'):
     removeDir('tmp_alert/')
   fname="tmp_alert/tmp.html"
   d = os.path.dirname(fname)
   if not os.path.exists(d):
     os.makedirs(d)
   f = open(fname, 'w')
   f.write('<!DOCTYPE html>\n')
   f.write('<html>\n')
   f.write('<body>\n')
   f.write('<p>Base Security Events</p>\n')
   #grab the list of headers
   x = getHeaders(baseEventList)
   f.write('<br>\n')
   #the baseTable class is a css file on the server
   #change if you have your own custom style sheet
   f.write('<table class=baseTable>\n')
   f.write('<tr>\n')
   #grab and write headers
   for i in x:
     f.write('<th> %s </th>\n' % i)
   f.write('</tr>\n')
   #loop over all events
   for event in baseEventList:
     f.write('<tr>\n')
     offset = 0
     #loop again to create matching value fields in our table
     for item in range(len(event)):
       itemwritten = False
       for header in x:
         if itemwritten:
           pass
         else:
           if header == event[item][0]:
             f.write('<td> %s</td>\n' % event[item][1])
             itemwritten = True
       f.write('</tr>\n')
   f.write('</table>')
   f.close()

# function to grab all base events using a list of ID numbers
def grabBaseEvents(xmlcontents, baseEventIDList, arcObj):
   #takes the dom document object for a single xml and baseEvent ID list
   baseEvent = []
   allEvents = []
   #for each sec event
   for node in xmlcontents.getElementsByTagName('SecurityEvent'):
     attr_list = node.attributes
     #iterate over sec event attributs
     for attr in range(attr_list.length):
       if attr_list.item(attr).name == 'id':
         #look at all the event IDs
         for baseevent in baseEventIDList:
         #when we have a match, iterate over the current
         #sec event child nodes and append them to a base event list
         baseEvent = []
         if baseevent == attr_list.item(attr).value:
           for x in node.childNodes:
             if x.hasChildNodes():
               if x.nodeName == 'destinationProcessName':
                 arcObj.setAttr('TPN',x.firstChild.nodeValue)
               baseEvent.append((x.nodeName,x.firstChild.nodeValue))
         #when we're done with the base event, append that to an
         #element in the allEvents list
         if len(baseEvent) != 0:
           allEvents.append(baseEvent)
   return allEvents

# Object to store and retrieve RuleFire Event values
class arcObject:
   def __init__(self):
     self.AlertName = "deviceCustomString6 not filled in"
     self.AA = "None"
     self.AH = "None"
     self.AU = "None"
     self.TA = "None"
     self.TH = "None"
     self.TU = "None"
     self.rulename = "None"
     self.time = time.strftime("%Y-%m-%d %H:%M:%S",time.localtime())
     self.filename = "None"
     self.targetproc = "None"
   def getAttr(self, attr):
     if attr == 'AA':
       return self.AA
     elif attr == 'AN':
       return self.AlertName
     elif attr == 'AH':
       return self.AH
     elif attr == 'AU':
       return self.AU
     elif attr == 'TA':
       return self.TA
     elif attr == 'TH':
       return self.TH
     elif attr == 'TU':
       return self.TU
     elif attr == 'rulename':
       return self.rulename
     elif attr == 'time':
       return self.time
     elif attr == 'FN':
       return self.filename
     elif attr == 'TPN':
       return self.targetproc
     else:
       pass
   def setAttr(self, attr, value):
     if attr == 'AA':
       self.AA = value
     elif attr == 'AN':
       self.AlertName = value
     elif attr == 'AH':
       self.AH = value
     elif attr == 'AU':
       self.AU = value
     elif attr == 'TA':
       self.TA = value
     elif attr == 'TH':
       self.TH = value
     elif attr == 'TU':
       self.TU = value
     elif attr == 'rulename':
       self.rulename = value
     elif attr == 'time':
       self.time = value
     elif attr == 'FN':
       self.filename = value
     elif attr == 'TPN':
       self.targetproc = value
     else:
       pass

Essentially I will parse the xml file for a correlation event, then parse the event for specific fields that I will store in the arcsight object. I'm thinking I'll use these fields later to fill in json files that I can post to the webserver. But for now I just store them and then grab the base event ID's associated with each event.

You may have noticed the deviceCustomString6 message I apply to the default alertName variable. That is something I have ArcSight set before it exports the rule-fire. I use that field to store the rulename + time + attacker address and a number of other useful things. The reason was that our ticketing system only lets us query on that alertName field. This way we can search for any of those fields (i.e. all rules triggered by a particular host).

Then I take those ID's, create a list of base events, using my grabBaseEvents function. Then I run that through my createHtml function which will create a table with the base events. This HTML file will probably be zipped up and posted to the web server as well.

Here's what the main loop looks like so far. The addNSend function will be what ultimately adds the fields to the json file and POSTs it to the server.

dir = glob.glob('*.xml')
for xmlfile in dir:
  contents = parse(xmlfile)
  #print "parsing " +xmlfile
  #for each Security Event
  for i in contents.getElementsByTagName('SecurityEvent'):
  x = i.attributes
  #if external ID attribute exists, it's a rule fire
    if x.item(1).name != 'externalID':
    arcObj = arcObject()
    arcObj.setAttr('rulename', x.item(2).value)
    baseEventIDs = []
    #map rule fire values to alert web fields
    for node in i.childNodes:
      if node.hasChildNodes():
        if node.nodeName == 'fileName':
          if node.firstChild.nodeValue != x.item(1).value:
            arcObj.setAttr('FN', node.firstChild.nodeValue)
          elif node.nodeName == 'deviceCustomString6':
            arcObj.setAttr('AN',node.firstChild.nodeValue)
          elif node.nodeName == 'sourceAddress':
            arcObj.setAttr('AA', node.firstChild.nodeValue)
          elif node.nodeName == 'sourceHostName':
            arcObj.setAttr('AH', node.firstChild.nodeValue)
          elif node.nodeName == 'sourceUserName':
            arcObj.setAttr('AU', node.firstChild.nodeValue)
          elif node.nodeName == 'destinationAddress':
            arcObj.setAttr('TA', node.firstChild.nodeValue)
          elif node.nodeName == 'destinationHostName':
            arcObj.setAttr('TH', node.firstChild.nodeValue)
          elif node.nodeName == 'destinationUserName':
            arcObj.setAttr('TU', node.firstChild.nodeValue)
          elif node.nodeName == 'destinationProcessName':
            arcObj.setAttr('TPN', node.firstChild.nodeValue)
          elif node.nodeName == 'managerReceiptTime':
            arcObj.setAttr('time', node.firstChild.nodeValue.rstrip('.0'))
          #check for baseEventIDs
          elif node.nodeName == 'baseEventIds':
            for ref in node.getElementsByTagName('ref'):
              attr_list_obj = ref.attributes
              for attr in range(attr_list_obj.length):
                if attr_list_obj.item(attr).name == 'id':
                  baseEventIDs.append(attr_list_obj.item(attr).value)
                  #find base events and create html to store
                  #grabBaseEvents should return a list of key/value pairs for
                  #each event
            listOfBaseEvents = grabBaseEvents(contents, baseEventIDs, arcObj)
            #createHtml creates the tmp_alert directory and the html
            createHtml(listOfBaseEvents)
          else:
            pass
      #addNsend(arcObj)
  addFileToFolder(xmlfile,'/var/tmp/processed')
  os.remove(xmlfile)

All in all, it works pretty well. I still need to make some formatting changes and add some logic to parse deeper into the xml for some relevant fields, but that's not difficult. Just time consuming. For now it's fine with the handful of fields that it parses for. I guess I could also have it pretty up the html file a bit. As it is, the headers are still smashed together field names like sourceAddress, and what-not.

Maybe when I have it complete, I can post it to ArcSight 724. No telling how many other people are in need of an API for a non-supported ticketing system. With a little modification, I bet this script would work for quite a few other systems.

-Newt