Jamie's Blog

The ramblings of a programmer with a little too much time on his hands

Author: GaProgMan (Page 1 of 19)

Discworld Disorganiser Header Image

Announcing: Discworld Disorganiser Web App

Today’s header image was created by Josh Kirby, you can find the more information on John Kirby here

A New Application?

I’ve wanted to build, publish and release a new application for a while now, one that’s based on an idea I’ve had bouncing around my head for a while: A search engine for Discworld novels.

Enter, the Discworld Disorganiser.

The Name

In the Discworld novels the dis-organiser is a parody of a PDA. It’s a small device, powered by an incompetent imp, which annoys the user. Not into remembering their schedule, in fact most of the features of the Mark 1 dis-organiser are apologies. It just annoys the user, that’s all it’s really good for.

The Application

The name doesn’t really hold with the aim of the application, if I’m honest. It was just a catchy, related name for the thing.

Anyway, the idea behind The Discworld Disorganiser application is that a user can use it to search for a Book, Character or Series.

The Book search would take into account as much of the book content as would be possible, when searching. Things like:

  • Name of the book (which seems quite important)
  • ISBN
  • Book description
  • Characters included in the book

That last step would be tricky, especially with the later books. Snuff (one of the final few books in the series) has over 50 characters, for example. Now most of these characters are not that important to the story, but I wanted to be as loyal to the source as I could in as many places.

A few weeks back, I released a video of an early beta of the app. It didn’t have a full data set, and was missing the Series search functionality, but I’ll include it here for completeness:

A Walk Through the Application

Since releasing that video, I’ve added the Series search function into the mix but have introduced a few bugs along the way.

First an updated video showing off the new features:

The first thing I do is search for a Book, you can see that it uses character names (which is why I search for “Rincewind” and “Nanny Ogg”) along with the book description (why I searched for “Watch”).

Then I search for Vimes on the character screen and we get the first bug: There are quite a lot of entries for Samuel Vimes.

Whoops

Then I do a partial character name match: “rince”. This matches on a few character names, so we get a few results. Again, though, there are too many Rincewinds.

And finally, I do a Series search. Currently this searches only on series names, so you wont get results if you search for “Nanny Ogg” or “Night Watch”. But you do get results for a series name, or for a blank string search.

Wanna Try?

If you’ve gotten this far, then I’m sure that you want to try it. And you can, it’s available for free. All you have to do is point a web browser at: http://discworlddisorganiser.azurewebsites.net

It’s on a free tier of Microsoft’s Azure hosting platform. This means that when the application isn’t being used it shuts down, and can take up to 4-5 seconds to start back up again.

Luckily, this only happens after about half an hour of no one using it

Wanna Know How I Did It?

If you got this far, then I know you’re more than a little interested in how I built this thing. Well you’re in luck, because I wrote up every step that I took to build both parts of the application.

Technically, it’s two applications

The first part of the application is a .NET Core WebApi project which uses Entity Framework Core and Sqlite. There was very little documentation out there for Entity Framework Core when I started building this application, so it was all a bit touch and go. But I documented the process, and you can read all about it here.

The series of blog posts presented in that link are in reverse order, because I like to mess with people.

The second part of the application is a .NET Core MVC Single Page Application using Angualr2 and webpack.

now THAT sounds impressive

I wrote all about the process of building that part of the application here.

I write a fair bit these days, huh?

On top of all that, I’ve released the source code for free. This means that if you follow along with the blog posts that I’ve linked above, you’ll be able to build a version for yourself.

Including the bugs!

WebApi

I’ve tried to keep the design for the  WebApi part to be as open as possible. As long as your dataset is complete enough, you should be able to swap the seed data JSON files for any other set of books, characters and series data.

Want to build a Brandon Sanderson search engine? How about a Stephen King search engine? Both of those are easy enough. Just swap the data in the JSON files, rebuild the app and call /database/dropdata followed by /database/seeddata to see your whole new search engine.

The source code for the WebApi project is available here

WebUi

The UI project is equally as open. As long as you change the address used in the search methods to that of your book engine, then everything will work fine.

The source code for the WebUi project is available here

Thoughts?

There’s not much else to say other than, go give it a try and let me know what you think. Let me know what you think of the application (or if you find any bugs) in the comments.

Post 2534 Header Image

How I Organise my Blog Posts

Today’s header image was created by Elvis Santana, the original source for the image is available here

Organised?! You Hardly Ever Post!

So it’s been a while since I last posted.

I start all of my catch up posts like that, have you noticed?

Well, I’ve been busy. I really have. I have three blogs now:

I post something new on the .NET Core blog

which chronicles my journey in learning .NET Core

every week. This means that not only do I have to make sure that I know enough about a given topic to write about it, but I also have to have some code written which shows off the topic of that week.

As an example, I spent February and March of this year writing a series of tutorials to go from nothing to this. Along the way, I wrote every line of code, designed a database architecture and published an application to the Azure cloud platform.

All of that in just five tutorial posts. All five of which come in at about 10,000 words

It was based on a project that I’d wanted to do for a while, so most of the design work had been done on the back of envelopes and scraps of paper.

But I still had to write the darned thing, then come up with a way of dividing it into chunks and writing about it in a way which meant that it would be engaging enough with readers for them to try it out.

I guess it helped that I gave the entire source code away for free, too.

On top of that, I have a list of posts, that are either written by myself or my brother, which are ready to be published for the Waffling Taylor Boys blog. This is a blog all about retro games and our thoughts on gaming in general.

They usually go out at a rate of two per month, but I still need to plan, research (read: play all my old games), write and proof read.

The posts are usually quite small, but my posts on Samurai Warriors and the NES classic TMNT have both been over 1,000 words, and my soon to be published post on the first SCUMM based Discworld novel tips over to nearly 2,000 words.

Tools

I use quite a few tools to write my blog posts:

  • Trello
  • Google Keep
  • Google Calendar
  • WordPress
Google Keep

When I come up with an idea for a blog post, I’ll usually have my phone with me or I’ll be near my computer. When an idea strikes, I’ll log into the Google Keep app

which is free, by the way

and start a new note. I’ll usually give it a title like

Blog post on …

and tag it with the blog that it would be relevant to. Here’s an example

in fact it’s an early version of the keep item for this post

Google Keep item for this blog post

I don’t flesh out the points much more than that.

this is a note taking platform after all

From there, I’ll head over to Trello and create a card for it.

Trello

Trello is one of the great fremium task management apps. The idea is that you create a card for a task and move it from list to list as it moves through stages of being completed.

it’s just a virtual version of a kanban board, but it’s really effective

Each of my blogs has it’s own kanban board on there, which means that I can focus on one blog at a time. Having the one board to focus on at a time is easier to manage than looking at the boards for all of my blogs at once.

The layout that works best for me is to have the following columns:

  • Ideas
  • Planning
  • To Do
  • In Progress
  • Published

On the .NET Core blog board, I have a “scheduled for publishing” column. This is because I’ll usually be, at least, one week ahead of myself. So it’s nice to have somewhere for each card to go between In Progress and Published.

Trello board for this blog

A redacted version of the Trello board I use for this blog

After I’ve created the Google Keep card, I’ll create a Trello card for it in the Ideas column. I don’t usually do much with the card while it’s in this column. But every few weeks, I’ll go through and move a card or two to the Planning column.

Once a card is in the Planning columns I’ll flesh out the main points that were on the Google Keep card, adding links and check lists as I go. At this point, it’s a short list of the main points that I want to hit, with some web resources to help me get the point across.

At this point, I’ll start adding labels to the card.

Trello card with labels

Trello has support for colour blindness. As far as I’m aware, I’m not colour blind, but I do prefer gradients and patterns over blocks of colour for my labels.

I’ll add labels based on what the card is about, the above card is about the Security of one of my blogs, and the steps I took within the software to secure it more, so those are the labels that I chose.

Once I’m happy with a card and I’m ready to start working on it, I’ll move it to the To Do column. Basically, this column is a waiting area until I get the time to work on a card.

When I’m ready to start working on a blog post, I move the top card from the To Do column into the In Progress column.

The most important thing here is that there is never more than a single card in the In Progress column at any time.  Each board can have a card in its In Progress column at the same time, but the In Progress column for a given blog can only have one item in it.

The reason for this is simple and can be summed up by quoting Charles Emmerson-Winchester III:

I do one thing at a time, I do it very well, and then I move on

Doing more than one thing at a time, especially something like writing, can put you under a lot of stress.

ask any university student around the time that their dissertations are due

So I only ever work on one blog post at a time.

Only when the card is in the In Progress column will I start to write it. By this point I’ve:

  • chosen the topic
  • planned out the content
  • picked a header image
  • picked out tags and categories

For instance, here is the card for this blog post:

Trello - Blog Post In Progress View

As you can see, I’ve fleshed out the original idea, added a header image, and added labels.

Once a blog post has been published, I’ll add a direct link to the live post to the card then move it to the Published column.

I can’t show you that for this blog post, so I’ll show you the last thing to go live, which was a post on my  .NET Core blog:

Trello - Published Post Example

Google Calendar

Having the cards move about is all well and good unless you have some sort of schedule for moving them around and getting the work done.

The best way to explain my work schedule is for you to see it, dear reader. So here’s my schedule for this month (April 2017):

Google Calendar Blog Post Schedule

Not included here is socialising time (time with friends and family), chill out time (because they’re different), bleed over time (when one blog post takes longer than the allotted time to complete), publicising time, and sleep.

oh, and work

Every Monday, I work on a post for the .NET Core blog. More often than not it’s me doing a write up on some code that I’ve written or some project that I’ve gotten live for people to play with.

Tuesday is my night for working on something for the Waffling Taylors blog. This is usually typing up my thoughts on a game, series or some other topic related to retro gaming. I’ve usually spent time during the preceding weeks playing the game or discussing the topic with my brother.

Unless it’s something that my brother has written, in which case I’ll have an evening of not writing

Wednesday is proof reading night. I take the post that I’ve written on Monday and proof read it. I read through it slowly, and some times out loud

which is a great way to check punctuation

I make my edits to the text and start again. I keep doing this until I’m happy with the post.

Thursday is the night that my .NET Core blog post goes live. I’ll usually spend half an hour before it goes live making sure that the post still makes sense (I make very minor edits here – capitalisation, usually).

One the post has gone live, I’ll grab a link to it and put it on the card as a comment. Then I’ll move the card to Published and spend part of the evening publicising it.

Friday is when Waffling Taylors posts go live, but that happens during the day. They go live during my lunch break, which is when I publicise it. Then all I have to do in this instance is to grab a link to the published post, add it as a comment to the card, and move the card to published.

Saturday is a free day.

On Sunday I’ll take some time looking at this blog. Sometimes it’s a post, sometimes it’s maintenance. If it’s a maintenance task, then it’ll get done on all three blogs.

Scheduling

Before I even started my .NET Core blog, I’d taken the John Sonmez course on blogging.

Here’s a direct link to it.

I found it insightful and helpful in picking a topic and getting everything ready for the off.

The basic rules that I set myself as a result of taking that course where:

  • Set a schedule
  • (as far as possible) Stick to it
  • Have a large backlog of articles ready to go
  • Engage with your audience

There’s a lot more to it than that, and I’d recommend anyone who wants to get into blogging take the course.

psst. It’s free

After All That

I often wonder how I’ve managed to stay on schedule

and there have been a few times when I almost haven’t

but it’s just a matter of putting the effort in and trying to be ahead of yourself.

When I am writing a blog post, it’s rarely posts that will go out that week. It’s usually due to go live the week after, at the very least.

It took a lot of effort to get to this point, and I’ve only had to pull something out of thin air very quickly once. It was when .NET Standard 1.0 was officially released and Immo Landwerth produced a bunch of videos talking about it and what it was.

then again that’s gone on to be one of my most successful blog posts, ever. And I’ve been blogging since 2010

It’s even been cross posted on Medium and it’s even been (partly) translated for a Chinese audience.

I was quoted for a Chinese technology news site.

I’ve had to relax the schedule of this blog to make room for the other two, but I still love doing all three.

Do you write for a blog? If so, what’s your schedule like?

Just One More Thing

No, it’s not my Columbo impression… this time.

I’ll tack this onto the end of the post, because it doesn’t really fit with the theme of the rest of the post, but I wanted to get it out there.

This week, I finished putting together the beta of an application that I’ve been working on for a while: The Discworld Disorganiser.

It’s a search engine (of sorts) for Discworld books. Take a look at the video and tell me what you think of the beta

What I've Been Doing - Header Image

What I’ve Been Doing 2017 Edition

Today’s header image was created by Camille Kimberly, you can find the original here

 2017 Has Only Just Begun

Really I should have come up with a snappier title for this blog post, but I had no immediate ideas.

One of the fundamental laws of computer programming is that naming things is hard. That’s my excuse here, and that’s what I’m sticking to.

This should have been titled something like “what I’ve been up to since late 2016”, but I’ll shut up about titles now and get on with the actual content. I mean, that’s what you’ve come here to read, isn’t it?

New Horizons

Back on October 4th, I announced that I was working on a new blog and that it would cover all things .NET Core.

If you’re not sure what .NET Core is and you’re not a developer, don’t worry too much about it.

Since October 4th I’ve been putting up one post every week on that blog.

Er, which you can read here if you wish.

At the time of writing, that’s 21 weeks. Also at the time of writing, I have 22 articles posted. Those articles have not been short, either. Most of them have been around 2000 words, and have had code samples that went with them.

Some of them have been about a particular topic within .NET Core (like this one, for instance) while some have been multipart tutorials (here’s an example). I’ve even collaborated on something with a friend of mine, and written about it (here’s my article on his blog)

Zac’s article was posted to my blog, you can read it here.

Views? Who Cares About Views?

This next bit is a little braggy, so I apologise in advance.

That first month, I didn’t have many readers. Only around 900 or so.

Only 900?!

But at the beginning of November, the Google SEO juice kicked in. I started to get referenced in places, and was even re-blogged.

Re-blogging is when someone copies your content, shoves it onto their site, and sticks ads all over it. They get all of the SEO juice and ad revenue, and you get nothing.

I got the re-blogged article taken down within 24 hours, but it was still pretty cool to know that someone had noticed me and wanted to rip me off.

Even at the time of writing this article, .NET Core is still very new, so that’s probably the reason regardless of what my ego says.

Then I noticed that one of my articles

shameless plug, click here, shameless plug

had started to get picked up.

What I’d done is write about the .NET Standard, which was a brand new thing that month (kind of), so Google saw my post as a very good source of information on it and because of that I’ve had about 150 people view that page PER DAY since it was posted.

That’s crazy. And it beats the per day stats of my original tutorial posts for x264 and MeGui.

Which still do very well, thank you very much.

External Stuff

On the back of that, I was published on Medium. Ok, they’re re-works of some of the posts I wrote for the .NET Core blog, but it’s still pretty good, right?

They’ve also been read thousands of times. Eep. 

Oh I was on a podcast about programming, too. Episode 8, as well.

We all know that the first 10 episodes is where it’s at.

I even have a really cool Cynical Developer T-Shirt:

More Blogs? Sure Why Not?

On top of all of that, I’ve put together a new blog with my brother. The aim of The Gaming Waffles of the Taylor Boys is for the pair of us to write about all of the games that we used to play or look forward to playing.

There’s already an article on there all about Super Mario Bros. and one about Resident Evil 7 (which was published before the game was released)

It’s fun stuff, not serious video games reviews or critiques. Just us discussing our favourite games and why we like them so much.

We’re also looking to get some guests on the blog, too. That’ll be neat.

There aren’t that many articles on there as of yet, but that’s because it’s still a pretty young blog and we’re trying to pace the content out.

There’s no point spending a weekend writing tens of articles and publishing them all at once. That’s not how it works for blogs. Consistently often, that’s the key.

It’s definitely worth taking a look and keeping a keen eye on it as it grows.

Any Others?

Those are just two of the projects I’m working on at the moment, of the ones I’m allowed to talk about.

It’s all a bit hush hush, right now.

I’ll write about the others in time. But they’ll have to remain sneaky and secret for now.

Mr B the Gentleman Rhymer

Music Suggestions – Mr. B the Gentleman Rhymer

Previously when I’ve written about music suggestions, I’ve recommended bands based on an entire album, based on an interesting take on a genre or based on a single track. This time, I want to recommend Mr. B the Gentleman Rhymer for no other reason than I think he’s ace.

Can Explain What This Chap Hop Thing Is All About?

Imagine if hip-hop had been invented during in the Victorian era.

Is it alright that I’m a little bit scared?

It’s a bit like that, but with a lot more cricket.

Mr B the Gentleman Rhymer is a self proclaimed Chap-Hop Superstar. Back in 2007 he released a song called “A Piece of My Mind” which was about the, then new, UK ban on smoking in public places and was an open letter to (again, then) Prime Minister Gordon Brown.

… It was also a hip-hop song performed entirely in received pronunciation.

Or, as some folks call it, “that posh accent from Downton, innit Bruv”

Topics for his songs include (to name a few):

  • Philosophy (with his track “Guy Debord”),
  • The state of the music industry (“Brit School” and “The Crack Song”),
  • Sartorial and etiquette matters (“Hail The Chap”, “Beats,  Rhymes and Manners” and “Brushed Tweed in the Hour of Chaos”)
  • Love and Relationships (“Lady C’, “Sherry Monocle”, “Curtsey For Me”, “A Throughly Modern Breakup”)
  • The history of his beloved Hip-Hop (“I Invented Hip-Hop”,, “Hip-Hop Was to Blame After All”)
  • And even a few covers of classic songs (“Chap-Hop History” and “Songs for Acid Edward”).

All of this (and more) showing his wit, wisdom and ability to write rhymes about, seemingly, anything.

Including comparing the current UK political landscape to the classic British Sitcom The Fall and Rise of Reginald Perrin in the first track of 2016’s There’s a Rumpus Going On.

Long Players

There have been six Chap-Hop albums from Mr B at the time of writing, with one having been released a few months before this article was written.

You’ll see why I specified Chap-Hop in a moment.

  • Flattery Not Included (2008)
  • I Say (2010)
  • The Tweed Album (2012)
  • Can’t Stop, Shan’t Stop (2013)
  • Mr. B’s Christmas Album (2015)
  • There’s a Rumpus Going On (2016)

I would definitely recommend all of the above but if I had to pick just one for the new listener, then I would recommend “I Say” from 2010.

In fact, I’m listening to it while writing this blog post.

It has enough catchy tunes on there (with minimal back and forward references) to keep anyone coming back.

Definitely check out his newest album, too.

Just listened to Hermitage Shanks, as I proof read that part.

The reason I specifically mentioned Chap-Hop earlier is because Mr B has also released an “Acid Ragtime” album under the name Mr B the Gentleman Selector – Acid Ragtime: Chapstep Volume One (2014)

It was first publicly played, in its entirety, during a Thursday Night Show web-cast.

What is Acid Ragtime? Imagine early 90’s acid rave music, but built around samples of Noel Coward and other early British film stars.

He has also released two mini long players on BandCamp, under the name “The Major“. Both are quite short, at 6 tracks long, and labelled as “Terribly English Electronica”.

Did I mention that he writes and produces all of these himself?

Pretty cool, eh?

Rivalries

There have been a few knockers along the way

… you know, do-baders. Haters, as it were

As with all Hip-Hop legends, Mr B has had his fair share of rivalries. Mr B is no exception to that, having had a public rivalry with fellow Chap-Hop artist Professor Elemental.

The difference between Mr. B and the more famous Hip-Hop rivalries, is that the one between Mr. B and Professor Elemental was settled via “The Duel” (a track on Elemental’s 2012 album Father of Invention).

Where Does One Start With Chap-Hop?

I would say that you should start at the beginning, if only so that you can witness the greatness that is Chap-Hop History:

In fact, why not just head over to Mr. B’s YouTube channel and give them all a watch? It’s jolly good fun.

I have to admit that my favourite track with a video has to be ‘ Just Like a Chap’:

Keen readers will recognise where the header image for this blog post came from.

Social Stuff

Being a modern musical act means that you’re expected to keep in contact with your fans on Social Media, and Mr. B is no exception to this. He’s quite active on Facebook and Twitter, replying to messages with fans and posting images from his live shows. There’s a blog over on Tumblr and some demos and recordings over on his SoundCloud (including a short series of podcasts), MixCloud and BandCamp pages, too.

With all that going on it’s a small wonder how he finds the time to record new material, let alone go on tour, but he does both very regularly.

Final Thoughts

Seriously you should totally check out Mr. B’s stuff, even if you’re not a fan of Hip-Hop. There’s a set covers of classic UK rave and acid tracks:

And one of the only acceptable Christmas songs to be recorded (the other being Jonathan Coulton’s Chrion Beta Prime):

Definitely give some of his stuff a listen, go see one of his live shows (he’s exceedingly good live), or just catch him playing one of his Gentleman Selector gigs (usually radio broadcasts). You wont regret it.

stocksnap_4yos7quwrv

Passwords And How They Are Hacked – Some Background Information

Disclaimer: I want to start this post by saying that I am, in no way, a computer security expert. However, the details presented here are correct enough to give a background and starting point for anyone who wants to follow up and dig deeper into the fascinating world of password security.

Everything about our daily lives is going up to the Cloud

What’s the Cloud, again? It’s a marketing term for anything that’s an Internet enabled service.

Your email account? That’s on the cloud. Your Facebook account is in the cloud. Your bank details are in the cloud.

What’s the problem with this? Nothing, really.

Unless the company that has all of your details on file (in their cloud) has a data breach. And if it does, you’ll end up on Have I Been Pwned.

Well, not on per se. I mean that your email address will be searchable there.

I’m searchable on there (and I’m not going to go into the details, because both are completely egregious examples of being added to databases and services that I never agreed to being added to in the first place), due to two unrelated data breaches.

Have I Been Pwned example

Have I Been Pwned? Yes I have. Have You?

It’s definitely worth taking the time to see whether your personal details are out there (due to a data breach or leak), and the guy who runs the site is perfectly trust worthy.

In fact, he’s one of the industry experts on this kind of thing.

Data Breaches

The name should be easy enough to parse, but a data breach is when someone breaks into a secure system (say your Bank’s computers) and makes a copy of the data that is stored there (say, account names and balances) for their own nefarious purposes.

Data breaches are not a new thing. Ever since the idea of organised businesses was created, there have been competitors who have wanted to steal their ideas and information. Then computers came along and data breaches got easier. Then the Internet came along and they got even easier.

OK, computer systems have become more secure since the early 70s.

Because security is ALWAYS an afterthought it wasn’t baked into how computers or the Internet worked from the begining, and we’ve been playing catch up ever since.

In some ways they’ve become less secure. The best IT folks out there will tell you, more than likely through the promise of anonymity, that even the best businesses have issues with their security. Most likely due to the users to of the system, but not always.

Users are bad? Yeah.

What Did Users Do?

Kevin Mitnick is a person who became infamous in the late 80s and early 90s for his escapades relating to computer security. He had spent his teenage years hanging around with Phreakers. These days, Phreakers would be called Hackers.

These were people who had figured out that there was a test tone on AT&T phone lines (around 2600 Hz), and if you played it down the phone before dialling, then your call would be free.

It’s so famous within the “hacker” community that there is a magazine named after it.

Soon after figuring out that the 2600 Hz tone would get them free phone calls, they started to learn other ways to get free things. Mainly they were after free access to computers or BBS‘s.

BBS’s were what we had before The Internet came along. Accessing them required a computer (which were expensive at the time) and an unmetered phone line (again, expensive).

To get access to these, Phreakers would spend their time figuring out how to get into buildings, and guessing (and resetting) passwords. They came up with a bunch of techniques:

  • Dumpster diving (for passwords that had been written down, and then thrown into the trash)
  • Creating fake IDs
  • Talking their way into the building
  • Calling random workers, pretending to be an employee and asking for favours

As a side note: the movie Sneakers, whilst fictionalised and mostly fantasy, shows how to use a bunch of these techniques in order to break into systems.

All of this falls under the umbrella term “Social Engineering”

Social Engineering?

Imagine the situation:

You’re at work, sitting at your computer working on some big project. Your desk phone rings, it’s Dave from IT. He’s new and is calling round to introduce himself.

Hey, this is Dave from IT. I’m still pretty new here and Steve, my boss, has asked me to install an update on everyone’s computers. I could walk up there with it on a USB and install it on each computer in turn, but that would take hours he wants it done now. Could you do me a favour and let me run it on your computer from here? It’ll save me a lot of hassle.

You will? Awesome, thanks. I’ll need your username and password, and you’ll need to not use your computer while I do it  – maybe grab a coffee or something. I owe you a beer, seriously. Thank you for saving my bacon.

What if Dave doesn’t actually work in IT? Have you ever met Dave or Steve? Was there an email or announcement that there was a new person starting in IT. Did you even pay attention to the number that came up on your phone (most internal lines will be a lot shorter than external ones – although, these can be easily faked)?

You’ve just given access to your computer to someone that you potentially don’t know.

This is an extremely simple example, but it happens every day. Why? Because we have a need to be helpful. It’s why we hold doors open for people, or pick things up when someone drops them. Because we’re social creatures, and being helpful is polite and expected from us all.

For more examples of how this is done, go watch Mr. Robot. It’s a fictional TV show, but there are some real security professionals who work on the show and some pretty realistic examples of how hacking is done.

The “Hi, this is Geoff from Microsoft,” calls have been happening to my friends a lot more, recently. This is another example of social engineering and they usually have the same format:

I can see that your computer has a virus on it. I need you to go to this website, download some software and I can fix it for you, from here. I’m from Microsoft, remember.

No. This person, whoever they are, are not from Microsoft. They’re a crook, and are tying to social engineer you into giving them access ot your computer. The key questions to ask yourself here are:

  • How did they get my phone number?
  • How do they know my name?
  • Why haven’t they called anyone else that I know?
  • How can they see that my computer have a virus on it?

One of my friends once asked the guy who’d called him this last question. The response from the “Microsoft Engineer” was a scripted, “We have an application that sweeps the Internet looking for computers with issues,” sort of thing. But when my friend asked the “Microsoft Engineer” how they could do this, considering that he didn’t have access to the Internet he was promptly hung up on.

What Does This Have To Do With Passwords?

Passwords are a hot topic. Even sciencey web comic XKCD has weighed in on it: https://xkcd.com/936/

Even a quick Google for password related topics returns millions (if not billions of results). If there is so much advice out there, then why do we still need to learn how to make safe passwords?

Because passwords, like security, are almost always an after thought.

Be honest with me

I’ll never know anyway, seeing as this text.

How many of your online accounts have either “password”, “password1” or something similar as their passwords? How many of them have the same password as another account? Does your Facebook account have the same password as your Gmail account?

Here is a link to one page (of thousands) that contains the most commonly used, and therefore worst, passwords of 2011 into 2015

See.

The biggest problems with passwords are:

  • Password reuse – is your Amazon password the same as your Twitter password?
  • Password entropy

Password Reuse

Let’s say you have a Facebook, a Gmail and an Amazon account. Let’s also say that you used your Gmail account to create your Facebook and Amazon accounts. For the purposes of example, lets say that your Gmail address is something like “[email protected]

This means that your usernames for both Amazon and Facebook are [email protected]

Let’s say that I guess your email account password. Actually here’s the more likely example: let’s say you signed into your email using some public or work computer and forgot to sign out.

If I can get access to that email account, I know how access to your Facebook and Amazon accounts.

Even if your passwords are different for each of these services, all I need to do is reset your password and I’m in.

Not a big problem? Are you sure about that?

Facebook will have your real name, phone number, a list of your friends, your work place, your home on it, and a list of places that you have been recently.

If you’ve entered any of this data, that is. And, let’s be honest, you have.

Amazon has your real name, phone number, address, business address (if you’ve ever had anything delivered to work), address history, and credit card information.

Still not a big deal?

Here’s the more likely situation: You sign into Facebook on a public or work computer and forget to sign out. From the Facebook account, we can get to your email account, from your email account, we can get to your Amazon account.

Heaven forbid you use the same Gmail account for work, too. That was the biggest issue with the LinkedIn hack – since it’s used primarily by business folks, with their work email…

And all that because you used the same password for them all.

And none of this has even touched actively trying to hack into these accounts by cracking the passwords.

Password Entropy

The extremely short version is that the longer a password is, and the more random characters it uses (without any that repeat), the harder it is for a person or a computer to crack.

If you want to read more about how entropy is estimated, you can read the wikipedia article on it, here.

I’m also well aware that it makes it more difficult to remember too (more on that, in a moment)

When a computer is trying to break a password, it has two basic ways of doing it:

  • Brute Force
  • Rainbow Tables

There’s actually loads more ways to do it, but these are the most often used.

Brute Force Attacks

Brute Force is what most people do when they forget their password.

I’m sure that it was password12345.

Wait! It isn’t?! Maybe password123456.

What?! Maybe it’s password1234567.

Except that a computer can do it millions of times a second. Usually they’ll either use a dictionary attack or just start at the beginning of the alphabet and work up, adding a letter at a time, until they get in (or the system locks them out).

Dictionary attacks are basically when a computer uses a file which contains all the words from a dictionary (might be the English dictionary, or a list of common words and phrases) and tries each one in turn.

But most of the time they use lists of the most common passwords (like this one, which I linked to earlier)

Rainbow Tables

Rainbow Tables are a little more complex.

Your password will, hopefully, not be stored by the website, app, or service as plaintext.

Plaintext is what you’re reading right now, you don’t need any kind of decryption system to figure out what these words are. So a plaintext password might look like this:

p4ssw0rd

Whereas an encrypted version of p4ssw0rd might look like this:

5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8

That example uses SHA-1 to encrypt the password. SHA-1 is NOT secure and shouldn’t be used for securing passwords.

Because good websites only store the encrypted version of the password and check that against the encrypted version of what you type, there should never be a way for anyone to guess what the encrypted password actually is.

A good website will “hash” your password (put it through some kind of mathematical equation that will swap all of the characters with something else), but it will also use a “salt” (something that is added to the encryption to add a layer of randomness) before storing your password in their database.

When you enter your password on a login screen, it is passed through these hasing and salting algorithms and THAT is what’s checked against the record of your password in the database.

Well, it is if they don’t store your password in plaintext

Because going from the plaintext password to the encrypted version is a difficult thing for a computer to calculate, there are groups of nefarious people who have done the hard work for you already. What they produce is Rainbow Tables.

These are collections of the most common passwords (from other leaks) that are already hashed and salted using a range of different encryption methods.

The idea behind these is that, once you’ve made a copy of the website database, you can look at the password fields and figure out which encryption system was used. Then you can use a known decryption system (usually reverse engineered) against all of the passwords in the database, and you’ll have them in hours (versus days and weeks of brute forcing the decryption of a single password).

A very basic version of this was used by the Bombe to break Enigma: figure out the common words and phrases, then use that decryption key to figure out the rest of the message.

What Can I Do?

Very little.

Pretty bleak, huh.

If the folks who made the websites and services that you used have protected your password then you should be ok. But only if you use completely different passwords for each website or service that you have an account on.

Seriously, you need different passwords for different things. If I’m able to figure out your password to your Facebook and you use the same password everywhere, then what’s stopping me from logging into your iCloud.

That sounds familiar

In fact, go check one of your passwords here: https://www.grc.com/haystack.htm

Before you ask: I’ve checked, nothing is sent back to any kind of server or stored anywhere.

What that GRC link will do is tell you just how long it will take a computer to guess your password. In fact, here’s what happens when you give it “password”:

GRC Haystack Password

How long it would take a computer to crack the super secure password of “password”

 See.

 Unique Passwords

Your absolute best bet to reduce the chances of it happening to you is to always use a unique password. But to be able to do that, you should be looking to use a password generator.

There’s an awesome one over on codeshare, which is free to use and adhere’s to the OWASP password guidelines.

Seriously, go try that password generator out. It’s super cool.

Now that you’ve generated your super strong password, how are you going to remember it? Well, that’s where password managers come in.

The idea with these is to store all of your passwords in one encrypted file with a master password being used to unlock them. I’m not going to compare them, because greater minds than mine have done that for me.

And there are a lot of them out there.

But I will recommend two fantastic password managers:

What’s the difference?

Well, LastPass stores your passwords on the cloud and KeePass stores them on your computer.

Depending on whether you want to be able to access your passwords on the go, or just on your computer will define which of the two you could use.

So in conclusion:

  • Don’t use the same password in more than one place
  • Use strong passwords everywhere

A New Blog Appears

Really quick post tonight to announce that I’ve built a new blog.

Ok, built is probably not the right word. But you know what I mean.

So, I’m taking some time to learn about .NET Core and have decided to set up a blog specifically for that.

Mainly so that I don’t fill this one up with posts that most folks aren’t going to read.

So if you’re interested in reading about my journey into open source .NET development, then head over to https://dotnetcore.gaprogman.com

That’s it for now.

Except to say that I still have a post about passwords and ways to keep them, and your online accounts save.

A Small Victory For My Readers

About 4 years ago, Google started a crusade to get all websites to use HTTPS.

That means that the connection to a site that you view is secure and encrypted.

They’ve even gone so far as to start penalising websites that don’t use HTTPS. Which falls in line with their older announcements that websites delivered over HTTPS would get higher rankings.

This means that if you look at your address bar, there should be a green padlock.

Like This

Like This

I am in no way an SEO expert.

I’m not even an SEO pert

… What? That was funny. Ok, it was Fozzy bear funny. But even so.

 But Google pushing for HTTPS was always a good thing.

Why Are You Telling Us This, Jamie?

Well, it’s a weird way to announce…

If the shoe fits! Am I right?

Ignore him. Anyway, as of right now my blog (what you’re reading right now) and my website are both being served via HTTPS. This is all thanks to the amazing folks at Let’s Encrypt.

They have an amazing post on their site about how HTTPS and Certificate Authorities (CAs) work. At the very least, you should check that out.

Pitfalls

There’s a one caveat to my blog being served over HTTPS, and that’s that some of my articles link to non-HTTPS content (usually old embedded images). That means that your browser might either warn you about it, or just remove that content. I’m going to be taking a look at each post and fixing them where necessary, but this could take some time as there are over 170 published posts (at the time of writing) on here.

I also might have to change my theme, as my search bar doesn’t seem to be using HTTPS, so that’s causing a few issues.

.NET in new places

Over a year ago I wrote about how I was building an application with Mono, Xamarin and GTK# on MacOS (even though it wasn’t called that then). Well, since then the Microsoft .NET team had announced that they were open sourcing the .NET runtime and were re-building it for non-Microsoft operating systems.

Because I wanted to play with .NET Core, I started trying to figure out how I could use it on my *nix machines. Because new Microsoft are awesome, they very quickly put up tutorials on how to get the run time installed on your machine.

Shortly after then, they also started building their documentation pages for all things .NET Core.

Thankfully this doesn’t follow the dated MSDN model, and it generated on page load from the GitHub documentation repo

.NET Core

Time has moved on since the initial betas and RTM releases, and we’re at version 1.0 of .NET Core. So I headed over to the .NET Core website and installed in on my Mac.

After that, I was looking around for the quickest way to develop apps for it. The quickest way to do that is to pull up the console and run these commands:

This would create a directory called dotNetCoreApp and create an empty .NET Core console application.

Pretty good huh?

We Can Rebuild Him

So npm is a thing that exists.

If you’ve gotten this far, then I’m assuming that you know what npm is. If not check out this wikipedia article for some background information [LINK]

Going back to the .net documentation (linked above), theres an article about building a .NET Core application using npm and yoeman, which is incredibly useful.

Using yoeman, you can template a full .NET Core application (console application or ASP/MVC web application stack) in seconds. So that’s what I did.

All of this is an extremely fancy way of announcing that I’m starting some cross platform .NET Core application development. The prototypes that I come up with will be hosted on my GitHub account, which can be seen here [LINK].

At the time of writing, I only have one application in that repo. It’s not that exciting either. All it does it print a message to the console.

Console Application Screenshot

It really doesn’t do very much at all.

That’s literally it. Nothing else. Exciting, huh.

What has been added to this repo so far was written entirely using Visual Studio Code. Why Visual Studio Code and why not something like, say, Atom?

Because Visual Studio Code is stupid fast and has all manner to plugins specifically for C#/.NET development (things like NuGet search and install).

The best laid schemes o’ mice an’ men, Gang aft agley

So what am I planning? Other than just learning the caveats and pitfalls of using .NET Core, I want to build a selection of application in C# with .NET Core. Some of the things I want to build are some MVC sites, and a few quick and dirty console applications.

The big one, however is medico. medico is an application that I’ve been designing for a long time, and I think that .NET Core is the way forward for it.

The design documentation is still in flux, and I’m still adding to it, but you can read the latest version of the medico documentation here [LINK]

So this means that, going forward, medico’s development will be restarting.

There are a few things that I need to look up first though. Things like opening files from disk in .NET Core (I know that the FileStream class wasn’t available back in the early beta). Once I’ve managed to figure out how to do these things, medico’s development will begin again in earnest.

Watch this space, I guess.

C# Exam – Threads

This post will contain my notes on the C# System.Threading namespace. For an extremely basic introduction to Threads, see my previous post [LINK]

Thread Class

The Thread Class is a found in the System.Threading namespace and can be used to create new Threads, get their status and manage their priorities.

The following is an example of how to use the Thread class.

This is probably not the most useful example of how to use a Thread, but it’s an example.

We create a Thread (called t), and give it a method to run (SomeThreadedMethod). Then we tell the t to start processing, then we tell Main() to wait until t has finished before moving on.

We use the Thread.Sleep method in SomeThreadedMethod to tell Windows that the thread that it is running in has finished. Without this, the thread would continue to run until the Windows scheduler has closed it’s time slot.

Thread Types

There are two types of Threads:

  • Foreground Thread
  • Background Thread

A foreground thread is used to keep an application going, its the thread that powers the GUI for an application. Google Chrome’s browser window is an example of a Foreground thread.

A background thread is meant for processing data without locking up the foreground thread.

Again, another gross over simplification coming

Imagine that you’re downloading a file with Google Chrome. There are (among other foreground and background threads) two Threads in use here:

  • 1 Foreground thread keeping Chrome’s browser window responsive (opening new tabs, scrolling up and down, etc.)
  • 1 Background thread performing the download action

The file download is performed on a background thread, because we don’t want Chrome to become unresponsive while downloading the file.

In .NET and C#, as soon as all Foreground threads are finished, the application that spawned them will close.

Technically is the CLR (Common Language Runtime) that closes the application

Passing Data

To pass data to a thread, we do the following:

This extremely simple example created a new thread using the ParameterizedThreadStart option which tells the Thread that we want to pass it some data when we start it. Then we do just that, we pass it a string to output to the Console when we start the thread.

A short note on the Console:

The console uses synchronisation to allow us to output to it from different threads.

If we wanted to output from the Main thread whilst t was running, then the Console would wait for t to finish writing to it before allowing Main to write to it.

Stopping Threads

The best way to stop a thread (without causing a ThreadAbortException) is to use a shared variable:

Threads have their own call stack, featuring the values of all local variables and any methods called during the processing of that Thread.

If you need to have multiple threads in an application then hand rolling them isn’t that great a solution. This is because you need to know, ahead of time, just how many threads you’ll need. This isn’t always possible, so to get around this, we use the ThreadPool class.

More on that next time.

C# Exam – Threads (Theory)

I’ve decided to take the Microsoft C# exam (code 70-483), mainly because I’ve been working in C# for 6 or so years now and it would be great to see how much knowledge I have of the language. Also, we’re all taking exams at work and I don’t want to be the one left in the dust.

Because of that, I’m going to be posting my revision notes here as I prepare for the test.

I don’t have an exam date yet, but I’ll be updating once I have one.

To prepare for this exam, I’m going to be using both of the following books:

Along with the Microsoft Virtual Academy series: “Programming in C# – Jump Start” 

That video series is free, by the way

Today’s topic is Threads and Parallelism, but first a little Computer Science for those who don’t know what a Thread is.

The following is going to be an over simplification of what Processes and Threads are. For a more in depth description, see the Wikipedia article on Threading [LINK]

Threads, Processes and Applications

Without Threads (more on what they are in a second), a single long running program or Application would chew up CPU time until it was complete. This would mean that your computer would be completely unresponsive until the Application was closed.

Imagine that you’re listening to an MP3 (maybe it’s a podcast, or maybe it’s just a song). Without threads, your computer/phone/whatever (delete as appropriate) wouldn’t be able to do ANYTHING else whilst the song is playing.

Lighting up the screen? Can’t be done. Moving the mouse pointer? It wont budge. Receiving a WhatsApp message? You’ll have to wait.

What modern operating systems have is a concept called a Process. A Process is a container around a single Application (in almost all instances). This means that every application on your computer/phone/toaster oven/whatever, is running inside of it’s own process.

Processes

A Process has it’s own Vistualised Memory (a virtual version of the memory in your computer). This means that each application running on your computer has it’s own memory space – so no two applications can read each others memory.

So, when you are running the application that shows you cat pictures and a separate application that’s playing a song, neither one can see the contents of the others memory. The music player can’t see the cat pictures and the cat pictures app can’t see the music that you’re playing.

This helps to avoid situations where one application will overwrite or corrupt the data used by another application. This was a common problem back in the early days of computing, because you had to manually map the memory space for your application (putting data in a space that you think wont be used by some other application) and you had no way of protecting that memory. Another application could come along and overwrite the data that you were processing in memory.

That’s a gross over simplification, but it’ll do for now.

If an Application, which is contained within a Process, crashes or locks up then only that Process crashes or locks up, rather than the whole computer.

If you’ve ever seen a program window go pale and your mouse pointer is replaced with a spinning timer, then you’ve seen a process lock up. Did you notice that you can still use all the other applications that are running on your computer, and even start new ones?

That’s Processes.

Well, technically it’s Threads.

Each Process runs on it’s own Thread. So what’s a Thread?

Threads

A Thread is a container for each Process. A Thread is similar to a Virtualised CPU, in that it’s completely self contained.

You can imagine a Virtualised CPU as an imaginary CPU that can be started and stopped by your operating system, and that’s (almost) exactly how they work.

The real (sometimes called Physical) CPU in your computer runs a thread for a short amount of time, before pausing it and starting another one.

Let’s say that you’re running a cat picture application, a music player and a file downloader. Your physical CPU might do something like this:

  • Start or resume the thread that the cat picture application is running on
  • Run the cat application for a short time
  • Pause the thread that the cat picture application is running on
  • Start or resume the thread that the music application is running on
  • Run the music application for a short time
  • Pause the thread that the music application is running on
  • Start or resume the thread that the file downloader is running on
  • Run the file downloader for a short time
  • Pause the thread that the file downloader is running on
  • etc

It does this constantly, giving you the illusion that your computer can run all three applications at the same time.

How long a thread runs for before it is paused depends on a lot of things, but it’s going to be in the somewhere in the hundreds of micro seconds for most desktop computers.

When a thread is Paused, it’s state (what the CPU was doing, the contents of the Vistualised Memory from the thread, and a bunch of other things) are saved before resuming another thread.

This process is called Context Switching, because the CPU is switching the current execution context (a cat pictures application, for example) for another (the music player application, for example).

Parallelism

Threads are what the operating system on your computer/phone/car/light bulb uses to utilise Parallelism.

Multiple threads (usually in the thousands) running on a single CPU is what makes this happen.

That’ll do for a basic intro to the theoretical background on Threads. I’ll post my notes for the Thread class in another blog post.

Page 1 of 19