For juvenile stupidity and classic Incipiat Turba nonsense, stick with Incipiat Turba.

For my computer nerd related comments, go to the new domain: ComputerMisuse.net

BSides:

Tim Keanini’s “Boyd’s OODA and General Predator/Prey Theory.”

I had actually wanted to catch Adam Ely’s “Exploiting Management For Fun and Profit – or – Management is not stupid, you are,” but I didn’t head over to the room early enough and the room for Track 3 was incredibly small.  TK’s presentation turned out to be very interesting to me.  In his talk he spoke about learning from predator/prey relationships in the study of Biology and relating them to the defense of computer systems. In discussions about information security or computer security, a frequent point is made that the defenders are disadvantaged because of their inability (for legal reasons or otherwise) to make offensive attacks.  From TK’s perspective, offense is not a requirement of successful defense.  He drew connections here to prey species which do not fight back with their predators, but survive by evasion. He then connected the evasion strategies to Boyd’s OODA Loop.  Basically, if the prey can loop through OODA faster than the predator, the prey can escape.  In technology terms, TK envisions a scenario where entire networks of systems using virtualization can be torn down and redeployed in short periods of time.  If servers are wiped clean and rebuilt faster than the attacker can gain footholds into the system, then the attacker will always be behind and unsuccessful.  My summary doesn’t really do it justice.  It was a very interesting view of the problem.  It’s a good counter to “we’re screwed because we can’t attack them back.”  Just because we can’t retaliate with similar force against the attackers, it doesn’t mean we have to sit in one place while they come after us. The Defcon presentation “Traps of Gold” by Andrew Wilson and Michael Brooks included some similar themes although their presentation did include attacking the intruders.

Schuyler Towne’s “Vulnerability Research Circa 1851″

I am a professional software engineer, so I attend these conferences to inform myself and talk about software security issues…  I am a novice lock picker and I’ve only recently started tinkering with locks for fun.  This presentation was still the most entertaining talk that I saw during the entire week.  It was an excellent recap of a time when there was great innovation in locks and lock picking.  More importantly, Schuyler pointed out the drought of discovery and increasing ignorance of the public and media since that time.  There are obvious parallels to the information security field.  Fortunately, with so many conferences going celebrating new ideas, we seem to still be in decent shape for moving infosec forward.  We must still be careful that we don’t let anyone take that away and go through decades of stagnation like the locks in Schuyler’s talk.  After all, there were still multiple talks that got pulled from BSides alone for one reason or another. Go give some money to the Electronic Frontier Foundation so they can defend our rights to explore: https://secure.eff.org/site/SPageServer?pagename=DON_splash

Defcon:

Moxie Marlinspike’s “SSL And The Future Of Authenticity”

Similar to my BSides experience, I watched one of my favorite presentations by accident.  I stuck around for this talk since it was sandwiched between the security research talk I attended and Dan Kaminsky’s talk, which I foolishly assumed I’d be allowed to stay for.  The gist of this one is that SSL is decent for ensuring the confidentiality and integrity of communications, but due to the reliance on Certificate Authorities, it’s not very good at authenticating the parties involved.  Browsers place trust in a static list of CAs to create valid certificates from now until the end of the internet, so if any of them get owned or are controlled by less trustworthy entities, then there’s a real danger of fraudulent certificates and man in the middle (MITM) attacks.  It’s possible to remove trusted CAs from the browser, but doing so may cause legitimate sites signed by that CA to break.  Moxie suggests a solution of creating “notary” servers to compare certificates.  Browsers would make a request to the notary server, which would request the cert from the site and send it back to the browser.  The browser will then compare the cert the notary received with the cert that it received directly.  If the two match, then things are good.  If they differ, it indicates a possible MITM attack.  In this situation, sites could sign their own certs because it’s not the certificate signing authority that is being trusted.  It is the notary server.  Users would have the ability to establish their own notary servers, or point to a public one.  With the current certs, you either trust the CA or you don’t. If a site has a cert from an authority that you don’t trust, then that site will be broken.  With the notary system, users could easily switch notaries without rendering a block of the internet inaccessible.  It’s a clever solution to a real problem.  I haven’t had time to play with it yet, but I intend to try it out.

updated 8/20/11 to add links:

You can (and should) watch the BlackHat version of Moxie’s talk here:
http://www.youtube.com/watch?v=Z7Wl2FW2TcA
and check out EFF’s SSL Observatory page, demonstrating how many CAs are out there:
https://www.eff.org/observatory

Tom Eston, Kevin Johnson, and Joshua Abraham’s “Don’t Drop the SOAP”

Whereas I enjoyed the heck out of Schuyler Towne’s talk despite it not connecting directly to my own professional work, “Don’t Drop the SOAP” was a big deal to me because of how closely it does tie in. I’ve bounced around a bit lately from being a “security tester” to a “load/performance tester” and now a “web services tester.”  I have to cover additional functional ground now compared to when my role was explicitly “security tester,” but I still operate with a security mindset.  I’m very interested in the testing methodology which they are working on.  It is being developed to fit into the Penetration Testing Execution Standard and will be included in the OWASP Testing Guide v4. I was hoping that there was something I was missing, and that they’d have some tool suggestions beyond SoapUI and BurpSuite (which I already use), but since the tool set is underdeveloped in this area, I’m excited that there may be opportunities for me to contribute.

edited 8/22/11 to add some more links:

WS-Attacks resource:
http://ws-attacks.org

Damn Vulnerable Web Service:
http://dvws.secureideas.net/downloads/index.html

 Some random thoughts from my notebook:

- The infosec community spends a lot of time talking about convincing businesses about security concepts.  They want to sell organizations on tools, secure development lifecycles, penetration testing etc.  Debates go on about understanding the business, and convincing the business to change the way they do things, because from the security community’s perspective, they’re doing it wrong.  Charlie Vedaa made some comments during his BSides talk “Fuck the Penetration Testing Execution Standard (PTES)” which made me think about this.  His statement was that from the business perspective, IT was tasked to build things.  They built them insecurely.  Now they are going back to the business for more money so that they can secure what they built.  That’s understandably a problem for the business. I think that’s the big problem here, that we’re trying to take insecure systems (and the people who build them that way) and force them into being secure.  As a quality tester, it is well known that the earlier bugs are found, the cheaper they are to fix.  We want to eliminate flaws as early on in the process as possible.  On some level I think that fighting with business about what they need to do for security is like fixing bugs found in production (we are literally finding bugs in production, but I’m talking about the process/behaviors here).  We should be devoting a lot more energy to education.  Computer Science and Computer Engineering programs aren’t adequately covering security, or even any other kind of testing for that matter.  I think some emphasis should shift away from fighting fires on old systems and trying to teach old dogs new tricks, towards making the next generation of IT professionals smarter about these things.  Right now, it seems like the curious few who will put it on themselves to investigate security issues and keep up to date.  We need a well trained majority to replace this curious few.

- Vulnerability Assessments vs Penetration Tests

I realize that penetration testing is super fun and all the cool hackers are doing it, but what about vulnerability assessments?  These seem to be left out of the talks.  Often I’ve heard them dismissed as “someone running a vulnerability scanner and handing over it’s output as the report.”  That doesn’t seem fair at all.  Obviously the behavior described lacks value, but comparing a poor vulnerability assessment to an advanced, well executed pen test is meaningless.  Part of it seems to be that everyone is talking about consulting engagements.  Perhaps it’s true that companies bringing in outside help for security evaluations should steer towards advanced penetration testers to get the most value.  As someone who is not a security consultant, but rather an in-house web application tester, what about my part?  I hear: some engagements will be like this, others like that, here’s how you obtain information about them etc.  I’m on the inside already.  I want to do better security testing, whatever that entails.  What do I do from here?

I am not a fan of the new logo and uniforms.  ASU is behind a lot of schools when it comes to traditions as it is.  I am very disappointed that you would trash our school colors and logo in a disgusting attempt to move some more merchandise.   I found the videos hyping the change to be offensive as well.  The message of “We’re changing, deal with it, you’re just whining because you can’t handle change” didn’t help me feel all warm and fuzzy about the new Nike State University designs.  I’m sure there are people on Eugene who think that football jerseys with tread plates on them are super cool, but the rest of the country thinks they’re a joke.  I’m not happy about my favorite team moving into the same category.

The thing is, I can handle change.  In fact, I could even get behind the addition of black to the jerseys if they weren’t so stupid.  Being Freaky Metal Kid, my closet is full of black shirts.  I’d love to have some Sun Devil gear in black, but this new stuff is no longer recognizable as Sun Devil gear.  ASU colors are Maroon and Gold.  That’s the name of the fight song.  The lyrics of “Maroon and Gold” include a line “Fight for the old Maroon (and Gold!).”  The black jerseys are noticeably lacking in Maroon.  Color-wise they look like Iowa Hawkeye or maybe Missouri Tigers jerseys.

They could have been ok.  In fact, you got one right:

The volleyball jersey has maroon and gold on it!  Softball looks fine too, although it doesn’t seem like a departure from ASU jerseys I’ve seen before.  If the black basketball and football jerseys correctly used maroon numbers with the gold trim, then everything would be fine.  It would still be possible to recognize them as Sun Devil colors.

After the colors, there’s of course the Sparky problem.  Sparky, the Sun Devil, is a unique and easily identifiable image.  Anyone who watches college sports can see Sparky and know that he represents the Arizona State University Sun Devils.  There are no others like him.  There are some other teams of devils out there, but none that would be confused with Sparky’s Sun Devils.  The pitchfork is not nearly so recognizable.  First off, ASU’s nickname isn’t the Pitchforks/Tridents.  When someone sees this logo, an obvious first thought is “Who are the forks?”  Secondly, there IS competition in this area.  Sun Devils and Blue Devils are different enough, but I don’t like how much this new logo reminds me of the UCSD Triton logo:

I think the new ASU version is actually a little worse because of that mysterious stuff on the handle.  Are those flames?  Seriously, what the heck? Again, I can accept change.  I understand that ASU has used multiple logos on the helmets over the years.  The idea of putting a pitchfork on there rather than Sparky actually makes sense to me.  The players are Sun Devils, so you decorate Sun Devils with pitchforks rather than images of other Sun Devils.  Sure, that makes enough sense.  Sparky doesn’t have to be the primary logo on the helmet.  He does need to be the logo everywhere else though.  Arizona State Sun Devils should be represented by a Sun Devil logo, not a flaming fork.
It is my hope that this will all go away.  It did once before.  Remember this accident?

That goofy disaster of a uniform lasted one season before our players went back to dressing respectably.  I really hope the same thing happens again.

Sincerely,

a fan of Sparky

When it comes down to it, I want a cert which will open up job opportunities for me.  Everyone agrees that a piece of paper does not automatically make someone highly qualified, nor does the absence of such a document mean that a person is not an expert.  However, I am at a point where I could use some evidence of competency to help me get my foot in the door.  If my skills are all they need to be, but hiring managers don’t see a convincing case in my brief work history, then I may not even get an opportunity to show off what I can do.

Based on that reasoning I decided to go looking for job postings which mention security certification in the requirements or desired credentials.  There are other certs out there which are not security specific that may still be beneficial for obtaining security related positions, but it would have required to much analysis to determine which should be included and which should be left out.  Below is a description and the results of my non-scientific search into security certification.  It’s my attempt to get an answer for the question “How many more job opportunities will I be eligible for by obtaining a given security certification?”

I started by collecting a list of the certifications offered by (ISC)2, EC-Council, GIAC, Offensive Security, and CompTIA.  I opted to search CareerBuilder, Monster.com, and LinkedIn for job postings.  I discovered after I started searching LinkedIn that in cases where there were no matching postings on LinkedIn that responses would automatically be returned from SimplyHired.  I decided then to include those results as well.  When I searched for each term, I simply typed in the letters for the cert and counted the number of job responses returned.  I did only a minor amount of review on the jobs returned.  If the first page or two of responses appeared to be relevant security jobs, then I accepted the numbers and recorded them.  In cases where the results were obviously dominated by unrelated job postings I attempted to modify the search to narrow it down to computer security positions.  I did not compare jobs found on one site to the jobs found on the other sites to ensure unique results, so it is likely that there are a lot of duplicates.  I did include searches for the names of the certifying organizations along with the certifications they offer.  Certain terms were problematic due to their overlap with non-cert related results.  Specifically “Disaster Recovery Professional” and “Network Security Administrator” may turn up many results which are not necessarily related to the specific certs I was searching for.

Once I had all of the numbers collected, I went through a couple tweaks to the spreadsheet.  I color coded the rows by certifying organization and then sorted based on the total number of postings found for each cert.  Due to the extreme difference in findings in SimplyHired compared to the other sources, I’m inclined to believe that it contains significantly more undesired job posts (false positives) than the other sites.  Since I didn’t investigate that too deeply, I left the numbers in.

Next, I removed the certification organizations from the table as well as a couple of the problematic search strings.  Eventually I ended up cutting down the list to the top entries.  I looked up some basic information about those certs and included that.

So what? CISSP is clearly the money cert (as I suspected going in).  I am interested in the relatively close grouping of SSCP (from (ISC)2), GSEC from GIAC,  and CEH from EC-Council.  The security architect at my company was very much in favor of CSSLP (based partly on erroneous information about the requirements), so I was surprised to see it turn up almost no results while the “worthless” CEH wasn’t far off in the comparison of the non-CISSP certs.  Obviously there are more things to consider about certification and things that could be done to make the data more reliable, but I don’t want to invest in certification that nobody cares about and I feel that I now have some idea of which certs people are looking for.

Professionally things got much more interesting.  I was given the opportunity to join the security testing team.  That has been a fascinating challenge.  After years of going back and forth I finally think I found what it is I want to do when I grow up.  Web application security is extremely fun and interesting to me, and I spend tons of my own time reading and experimenting with things because there is so much to learn and it’s all fascinating to me.  On the negative side, I’ve found that the organization is far from having the same enthusiasm that I do and I’m beginning to see the limits to what I’ll be able to accomplish in my current job.  I feel like I’m nearing the end of what I can learn and accomplish in my isolated corner of the office.  There is no leadership and my attempts at leading from the bottom have been repeatedly thwarted.  Several months ago I signed up to be a mentee in the Infosec Mentors project, but I was never paired up with a mentor.  That was disappointing.  I’ve started looking out for other job opportunities and have applied for a couple different positions.  Neither turned out to be good fits for me, but I did have a worthwhile conversation with the top security guys in another company and they gave me some good information which I think will be helpful moving forward.  One piece of advice that has bothered me regarding Infosec, is the suggestion that everyone should blog/tweet.  I feel very limited in what I can say because so much of what I’ve found, learned, and experienced is directly related to the company that I work for.  If I post “these are the security problems I keep running into” then I’m disclosing potential weaknesses to anybody who knows where I work.  It seems easier for consultants who can always say “customer x had these problems…” and the identity of the client can remain unknown.  One answer to this dilemma would be to write about more generalized technical issues.  At this point though I have only spend one year dedicated to security testing.  I’m not figuring out anything new at this point.  I’m finding flaws that are already well known (XSS, SQL Injection, CSRF etc).  My methods of locating them and exploiting them are based on information other people published a long time ago.  What can I really say?

The whole career thing has been really frustrating to me lately, so let’s move on from that.  Here’s the year end stats for Incipiat Turba:

The pages:

1)Types of Beards/Mustaches

2) Freaky Metal Kid: American Genius

3) Incipiat Turba

4) FMK vs Selenium

5) Wilcox Rocks (Larry Wilcox Tribute)

6) Lost and Found (angus young drinks through freenachos jerk)

7) Freaky Metal Words

8 ) Lost and Found (moving agitator pictures through timeline for kit kat bars)

9) Communist Bowling

The 10th page was some test page that I was working on, so all the hits there were me.  There was a big drop off after #9 anyway.

2011 Goals:

- Prepare for and take either the CEH or CISSP exam

- Get back to BJJ (I’ve been out since the beginning of May 2010!)

- Update the site/blog more (I’ve been saying this for over 10 years now and it pretty much never works out, but maybe 2011 is the year)

- Rock out more.  I barely go to concerts anymore!  WTF?  I need to fix that.

- Finish my Java projects and start learning Python

- Read Hacking Exposed

- Attend Defcon 19

- Make my communication skills less defective

- Get married

The most interesting thing in this election is the collection of bogus Green Party candidates on the ballot.  I heard about this when the primaries happened, but I didn’t pay as much attention to the details.  It’s amazing how fake they really are.

In this article: http://www.azcentral.com/arizonarepublic/news/articles/2010/10/05/20101005arizona-green-party-candidate-larry-gist.html you have Larry Gist refusing to answer any questions about his values.  The Green Party’s positions are clear, how hard is it to go on record as agreeing with them if you are in fact, a Green?  You could try to reach these candidates, but all you’ll get is a Starbucks: http://www.azcentral.com/news/election/azelections/articles/2010/10/03/20101003arizona-green-party-candidate-starbucks.html

The fun part is reading the candidate statements.  The entry for Theodore Gomez (Photo NOT submitted) says this:

“I was recruited to come to Arizona from Michigan by Collins College in Tempe.  The promised me that if I got myself to the school they would arrange for student loans and a job.  A friend and I loaded everything we owned into a car and drove across the country.  When we arrived the admissions counselor at Collins College enrolled us and signed us up for tens of thousands for dollars in student loans–but no job.  So now we are homeless and live on Mill Avenue.  It’s hard to study when you have no place to sleep, so we dropped out of school but Collins College won’t refund our money.  We aren’t the only students who have been scammed by Collins College.  They lure young people here with promises of a better life and then pile us up with debt that we can never pay and throw us out on to the streets where we resort to asking strangers for change.  If elected I will demand that private colleges be regulated so they can’t hurt kids any more.  Thanks Steve May for helping us out.”

That’s the case he’s making to be elected for public office.  Those are his qualifications and views on the issues.  Also, Steve May (thanked by the candidate above) is a Republican drunk driving enthusiast, so if he’s supporting a random collection of “Greens,” that’s a little shady.  He did ask them if they are fake and they said “No,” so maybe I’m being too cynical.

As you can see from the following data, I am far from conquering the beast that is the beard image:

Apparently people don’t care for any of the posts which took effort.  The site gets more attention when I upload a scan of something silly looking and just stop there.  Dang.

While we’re talking Google Analytics, I learned something recently.  It was yet another thing I wish Google had documented, but maybe I’m the only dope who didn’t already know it.  Here’s the situation: on the primary profiles (of the account I maintained at work, not Incipiat Turba) there is a filter which removes all of the query strings. There are hundreds of different parameters across all of the tracked pages, so it simply wasn’t feasible to remove them by entering them in the tiny textbox Google allots for them.  The filter uses the following regex on the URI field:

(.*?)\?

and in the constructor field overwrites the URI with $A1.  Things get tricky when certain parameters are determined to have value.  I end up writing additional filters to work around this one.  Advanced filters will be used to find the desired parameters and then alter the URI prior to the above filter.

somepage.jsp?paramToSave=value

will be changed to something like

somepage.jsp-paramToSave=value

That part is simple enough.  I ran into a complication when some other, unwanted parameters piggy backed onto the one I wanted to save.  I’d see things like:

somepage.jsp-paramToSave=value&junkParam1=crap&junkParam2=dontcare

It took me several tries to get the regex right to drop the other parameters.  I first wanted to use something like:

(.*paramToSave=)(.*)&

and then write $A1$A2.  Unfortunately, the second item is greedy.  When my filter was in place, I ended up seeing:

somepage.jsp-paramToSave=value&junkParam1=crap

That’s not good!  I know I learned about regular expressions in college, but I will admit here to totally forgetting them.  What I currently know about them is mostly based on my experience of writing Google Analytics filters at work.  That involved a lot of trial and error.  Normally I’m smart enough to create new profiles for testing them, despite that FAIL that I had in the picture above.  I’m not sure which page I used for a reference, but any GA regular expression pages have the same information as this one: http://www.google.com/support/analytics/bin/answer.py?hl=en&answer=55582.

A little note that I can stop the greed by adding a question mark would be nice.  I was confused and trying all sorts of unsuccessful bullshit until I figured that out.

(.*paramToSave=)(.*?)&

gets it done.

On a happier note, I finally found a more appropriate home for Google Analytics at work, so I’ve handed the reigns over and gave up my admin privileges.  It’s good to be free.

It’s also good to be free from Windows.  Many of my coworkers around me are still complaining about how Firefox 3.6.3 was pushed onto their machines (with a rather obnoxious configuration on top of that).  If you fix it.  They’ll push it back.  Rather than have my machine repeatedly crippled by the whims of some idiots in another office, I just ditched the standard Windows install for my own Ubuntu install without any of their dirty looks hooks or restrictions.  The only small issue at the start was replacing Outlook.  Thunderbird, Davmail and the Lightning extension handle that well enough.

Now that my tool set is no longer under attack and I’m not stuck on Google Analytics tasks, I have plenty of time to be an idiot about coding fuzz tests.  More on that soon.

- Blarg!

Why?  Why did they downgrade my browser? There was no warning.  No email saying “Hey, we’re pushing a browser on you, you might wanna back some stuff up.”  There was no choice to say “No thanks, I already have a more current version.”  It turns out that it’s part of the release process for a new HR page.  The standard image on our machines contains an outdated IE browser.  Some internal applications only function correctly with the sad old IE.  Now their big new update to the HR page malfunctions in IE, so what’s the plan?  Force Firefox onto everyone’s computer, whether they already have it or not.  In our building we’re talking about programmers and testers here, so everyone already had it.

I found out the reason for the Firefox push when a big crew of guys started approaching every cubicle on my floor to ask people if the new link on their desktop worked.  Mine did not as it couldn’t find Firefox.  During my paranoid reinstall in the morning, I installed 3.6.10 to a different directory in the hopes that any more random browser pushes would spare me.  I also created a non-default profile and backed it up once I got my addons installed again.  I noticed the desktop shortcut wasn’t opening in the default browser (Chrome), but trying to start Firefox from the default location.  The dude inquiring about the link indicated the problems with IE and said that they were going around now to check the links with everyone because they wanted to avoid calls to tech support when the HR page was updated.  Yeah, they screwed up everyone’s Firefox installs so they could avoid tech support calls from the few clowns who stick to the old versions of IE for their browsing.  Way to be efficient guys!

More than a recap of the game, which you can get anywhere, I wanted to mention how impressed I was with Camp Randall Stadium and the fans it contained.  My friends and I showed up to the game, wearing our best Sun Devil gear, shouting and cheering for ASU.  There were no confrontational responses.  Nobody shouted at us, harassed us, or threw things at us.  We were frequently welcomed with polite greetings and handshakes by Badger fans who generally said something like “Welcome to Wisconsin, I hope you enjoy the game.”  It’s sad, but that is not at all the greeting that I see at Sun Devil Stadium where people are more frequently rude and vulgar.  Since that’s where I’ve experienced most of my live college football, I expected a hostile response as I showed up to Camp Randall in the opposing team’s colors.  I was very impressed when I found that I was completely wrong.

Following the initial pleasant entrance, the fans continued to demonstrate how much better they are at being fans for the rest of the game.  The stadium was full.  It was nearly solid red (unattractive to me, but impressive).  The Wisconsin students sang, clapped, and danced with fantastic enthusiasm.  Several amusing variations of the wave went around the stadium.  The upper deck shook noticeably when House of Pain’s “Jump Around” was played over the PA and the entire building did exactly what the song suggested.  Late in the game, when Sun Devil Stadium  bleeds “fans” regardless of score, Camp Randall was still full.  During the game, a part of Wisconsin’s band came up into the ASU section.  They were initially greeted with boo’s, until Sun Devil fans figured out they were playing “Maroon and Gold.”  That’s very cool.

After the game we stuck around for the 5th Quarter.  It featured an extra twenty minutes of the band playing and dancing on the field and fans singing and dancing in the stands.  They again played ASU’s fight song.  I was reminded on several occasions that this post-game party happens after every game whether the Badgers win or lose.  It was very fun, and I’m totally jealous. When we made our way out of the stadium and began walking down the road towards State Street we ended up walking parallel to the marching band.  The band was still playing and dancing as they marched with police escort down the road.  It was entertaining the whole way.

When all was said and done, it was the most fun I think I could ever have during and after an ASU loss.  Wisconsin fans made me feel welcome at their big football themed party and I really hope that they are treated appropriately when they come to Tempe.

Thanks Badgers,

Freaky Metal Kid

Let’s start from the beginning…

A couple of years ago I was brand new and fresh out of quitting teaching (this post is about a figurative IT nightmare, I still have literal teaching nightmares).  My job was performance testing the applications at my company.  In order to accurately model the load, I had an obvious question of “What does the usage in production look like?”  The answer was “Nobody knows.”  The company is neither small enough nor new enough for that answer to make any sense, but I had to dig anyway “How can I find out?  Can I get some logs or something?”  I was then told to request access to the Google Analytics account, which supposedly held the answers.

I logged in a peaked around to find that it was in a state of serious neglect.  There were no filters in place.  Traffic was randomly segmented into different profiles using different tracking codes not for any logical reason but they just added the JavaScript at different times and didn’t know what they were doing.  Most of the traffic was recorded as the non-helpful “(other).”  I’d later discover that the JavaScript was even worse.  There were pages with multiple, sequential setVar() calls.  It appeared in some places that they were also trying to use setVar to differentiate multiple views on a single page.

I needed some real numbers and this seemed to be the only way to get them, so I spent a day or two reading some Google Analytics documentation.  I found some things that I thought would clean up the data a lot so I started shopping them around.  Every person I talked to told me to talk to somebody else.  Eventually I found a person who worked with the usability group.  She wanted numbers as well.  She didn’t have any experience as a Google Analytics administrator, but she had admin access to the account.  Through her I was able to get some simple filters in place (everything to lowercase, removing the query strings, identifying default pages being reported twice as the directory root).

As things began to clear up, more problems would become apparent.  Eventually my admin pal got tired of me telling her to add filters, so I was granted access so that I could add them myself.  Certain problems took longer to figure out than the initial issues, so I did more reading.  You probably see my mistake already, but I was new, I didn’t.  I got some help from Justin Cutroni by posing some questions in the comments on his site.  I read Brian Clifton‘s book.  I read Avinash Kaushik‘s book.  I also got my manager to send me to a Google Analytics Seminar for Success.  My primary work responsibility was still testing software, but tasks came around sporadically at that time, so I didn’t mind filling in the down time with the analytics stuff.  I was a math nerd before I was a computer nerd, so the wall of statistics was interesting.

My Google Analytics pal from the usability team and I went through and cleaned up the data as much as we could.  Many new profiles were created, lots of filters were added.  We pushed the development teams to correct the JavaScript.  We figured out what a deceptive little goon setVar really was.  Seriously, what the hell, Google?  It caused interactions to be recorded which completely subverted the filtering on different profiles.  The visit numbers would show thousands of visits and no page views.  Through testing we figured this out, but the documentation provided no good explanation about the side effects.

Then people started coming to us with questions about obtaining more data.  How should they implement the JavaScript?  Could we filter X?  On the technical side, this was perfectly easy.  I was capable of telling anyone how they could obtain (if it could be done) the data they wanted using Google Analytics.  The problem for me was/is that I am at the complete bottom of the org chart here.  I don’t make business decisions with the data.  I know people do.  I’ve heard them talk.  I cringe when I hear GA numbers being thrown around.  Numbers that are clearly wrong.  Numbers that don’t mean what they think they mean.  Since I am not in touch with how the business wants to use these numbers, or what data is relevant to them, it’s difficult to answer GA questions which will determine the data available for everyone.  It’s tricky because there’s no going back.  The data cannot be parsed again or filtered differently.  Also, our site spans several applications and multiple domains and sub domains.  One development team’s poor handling is capable of wrecking the numbers for the others.

I’ve been trying to express this.  I’m a tester.  I’m not even a performance tester anymore.  I have no need for the numbers myself.  I have no direct line of communication to any important decision making people, but I control the data.  The numbers they see are the ones I configured it for them to see.  I wrote a whole bunch of regular expressions that move a bunch of strings around.  Nobody else verified them.  Nobody else knows what they do.  I documented them and pointed people to them.  I tried to do presentations to explain what I’d learned so that someone else would be capable of maintaining the account.  At the very least they should double check the things that I’ve done.  The result of the presentation was pretty much “It sounds like you know what you’re talking about.  How about you just keep it?”  D’oh!

As my new testing responsibilities have been more consistent than my old ones, I’ve had no time for analytics.  I tried to bring it up, but I was told that they’d be switching to Coremetrics because GA doesn’t do what they want (like they have any clue what GA does).  Allegedly there exists someone in the organization qualified and willing to be the steward of Coremetrics.  Months, then years go by and people keep dumping more and more into Google Analytics.  Coremetrics is still not a significant part of their data collection.  I peak in every now and then and see the once tidy organization, falling back into the state of chaos which existed back when I first found it.  In one place they are using it as a security audit log.  I hope your eyes didn’t roll right out of your head when you read that.  Mine almost did.  The head of product development is ok with that though.  I spoke with him directly about my data concerns.  They’re switching to Coremetrics… I don’t need to worry about it.

Recently I got a new manager.  On my behalf he found a new home for Google Analytics maintenance.  Supposedly, I’ll finally be handing that responsibility to a more suitable person in the near future.  As though they sensed it coming, one of the dev teams I’m supposed to do testing for has requested a bunch of changes to the filtering in GA.  I haven’t had any time to do the knowledge transfer, so I’ll have to do it.  Just when I thought I was one step from done, the end moves back from me.

Nobody wants to own it.  Everyone wants the numbers.  Nobody knows what they mean.  I can’t believe this is a real business and not The Office, Office Space or Dilbert.