An inspiring video – Mick Ebeling’s story of hacking technology

Posted on Updated on

Mick Ebeling’s story of hacking technology.

I saw this video a long time ago, and bookmarked it for sharing.

Do you also hate the new Android YouTube app layout?

Posted on Updated on

Lets be clear on something from the start: I really like YouTube. I use it all the time, almost daily, and I really enjoy it because there is such a wide variety of content on there to consume.
I also really like and appreciate how the YouTube ecosystem has spawned a lot of independent channels dedicated to producing quality content. Anything from daily news, documentaries, independent movies, comedy, tutorials, advice, education… so much. Read the rest of this entry »

Microsoft Azure and Amazon Web Services comparison

Posted on Updated on

Well, this is interesting…

Microsoft has added what appears to be a very comprehensive comparison of services provided by Azure and AWS.
What is interesting is that there are (currently) some services provided by Azure that dont (appear to) have an AWS equivalent.

With Microsoft seemingly making positive mindshare strides in recent years, this is another good move it seems.

Details can be found here:

A temporary fix for Logstash S3 input authentication

Posted on Updated on

A while ago I ran into an issue where I couldnt use Logstash and the ‘logstash-input-s3’ plugin, and the manual authentication method didnt work well.

Read the rest of this entry »

Process AWS ELB logs with Logstash! update!

Posted on

I previously wrote about making a grok filter to process ELB logs.
I have since worked on this further and developed an updated filter, which has been working very well for some time now.
As there were some comments on the previous post, I thought I should upload my working copy that I use right now also.
Read the rest of this entry »

Process AWS ELB logs with Logstash!

Posted on Updated on

I recently ran into a problem with Logstash, where it would not correctly and consistently process AWS (Amazon Web Services) ELB (Elastic Load Balancer) logs from an S3 bucket.

ELB logs are the standard method of logging ELB usage and activity in AWS, and are very useful when trying to determine a wide variety of trends and activities.

I was determined to get ELB logs into Elasticsearch, using logstash, so I needed to figure out a few things… Read the rest of this entry »

Fixing permissions on CloudTrail S3 objects

Posted on Updated on

Where I work we aggregate all of our AWS CloudTrail logs from separate accounts into a single S3 bucket in a central account.

Yesterday, I ran into a weird problem where I noticed that our Logging solution, ELK, would not process files dating before a certain time.

Upon further investigation, I discovered that the date period missing was before we moved all of our existing files into the bucket from the AWS account it was originally in. So, “simple!” I thought to myself, “I’ll just update the permissions and allow ELK access to the files!”.

I was wrong, not simple. I had to fix this anyway… Read the rest of this entry »