Author: Kenneth

How to get a big break – Notes to my younger self.

How to get a big break – Notes to my younger self.

Getting into the positive territory of my talk! A turn for the worse, ended up being a turn for the better! Why, how, this is what happened and how I got my ‘big break’.

This is a direct carry on from the previous post and is as the others have been based on my talk that was given at the Finland React JS conference. If you’ve read the previous post, it wasn’t a terribly happy ending. I did create a good number of items, but over time that one main item which was a failure ultimately meant that more work wasn’t coming through. Due to previous experiences, what I could see happening and also the fact that I was really enjoying messing around around with Adobe Flex/Actionscript I took matters into my own hands.

Direct Action

Work was drying up, getting zero direction from management, even after asking about creating work for the client that I knew they could use – after all why not give them something when I wasn’t working on anything. Nope…

So time to take matters into my own hands. I was busy doing little demos, experimenting with new libraries, new techniques, answering questions on Stack Overflow, while also reading some common questions and creating examples based on those questions. What did I do with these demos/snippets? Well I created a simple WordPress blog (yes this blog) and every now and then I’d put up my examples and source code.

This wasn’t done to get coverage, but purely to help share my knowledge. In turn my actions on sharing my knowledge allowed me to extend my knowledge. My site way back then used to be rather popular – I even made a little bit of money at it via Google ads and some free merchandise! Which was a nice bonus when you expected nothing.

The Point

What am I getting at. Well, by experimenting, keeping things small and simple allowed me to learn the intricacies of the language, learn the lifecycle of the flashplayer, find out where the memory leaked, the add-ons, as well as the latest and greatest releases. This really mattered.

In todays market, this would be like me taking the latest hooks coming out in React 18 and creating demos against each one. Comparing it to what went before. Figuring out where and when lifecycle methods were being called and why. So dig in, create the smallest of demos and tinker around.

The Breakthrough

First place I worked while in London

So not to far down the line, the inevitable happened and redundant once more. Even though it wasn’t the first time and not a huge surprise, it doesn’t make it any easier. While hunting for jobs, there were zero in my local area. Quite a few down south in London and in the end I got hired by a high end consultancy company who worked with a range of leading banks.

While I can’t say for sure why they exaclty hired me, I know my blog was a big selling point of my coding abilities (yes, it’s very much lapsed now – this was around 15 years ago) and when it came to the interview as I now knew the code framework inside and out, then that certainly got me through multiple stages in the interviews.

In the end, each week I’d fly to London on the Monday morning, and fly back home on the Friday. I was renting a place in London while there and even with those extra expenses I was still coming home with way more than I’d ever done in the past – not to mention it was great fun being involved in such a large project. But there was a cost, but that’s for another story…

Now that I’d entered the contract market, that gave the outlook on doing work a completely different feel. Again that’s for anther story. Something that for those that can, I’d certainly recommend as long as you are aware of it’s drawbacks.

Works on my machine! Notes to my younger self.

Works on my machine! Notes to my younger self.

Thanks for that, good to know – Works on my machine! It’s such a common issue, hey lets ship your machine then 😁 How did we get to this situation. A lack of Ownership & Trust, that’s what! Part 3 from my series on the talk I gave at React JS/Finland. You can watch the talk directly here – https://youtu.be/zE8PDM_7xoQ?t=11404 This part covers my discovery that coding, isn’t always about coding. Read on…


For this one, I need to set the scene to highlight what I did and why it was wrong. Being more experienced now, I certainly wouldn’t have done what I did all those years ago.

The story

I was in a new role, one the main reasons for me being in that role was to fix an application that someone else had created that wasn’t functional. This application was created by a web developer who hadn’t written any web applications before. The process of designing/creating a web page is drastically different to creating a web application.

So I come in to this role, get shown the application, asked my views on it and how to improve it. After a while digging into it and running it, my conclusion was that I needed to re-write it from scratch. So that’s what I said. Like would I not say that – after all that was what was needed!

The answer to my request was a No. As it was my manager that said no (also the same person that wrote it) I was OK, fine I’ll get to work with trying to fix the issues. And yes I did improve it, I got it down from rendering a view in around 2 minutes, to being usable in under a second! I was like brilliant! Every time you make a large leap forward in getting that time down was like a huge win. So all was good – yeah?

No it wasn’t all good

I’d gotten the app down to an acceptable speed, pushed to production, then got the client to use it. They were still not happy? Why on earth were they not happy. It worked just fine (on my machine!). Anyway after a call, I go up to the clients office (for the first time), and within moments of being in the office it was clear why they still had issues. Their machines were archaic, super slow, well out of date machines. There was next to nothing left to optimise now, not a great situation.

What went wrong?

Firstly I should have understood that even though the guy that created the application knew it was rubbish, it was still his rubbish. It was his personal thing and as such someone coming in and suggesting it be started from scratch wasn’t going to fly. People don’t like to be corrected, even when they know that they are wrong. It’s human nature. You have to work at providing the evidence in a clear and balanced manner. Not just – it needs scrapped. Must understand the human element of coding.

Secondly, I should have found out what the client wanted. I should have found out what type of system the user was going to use the application on. After all the first pass of the app was out. The client may have loved it’s look and feel, or equally they could have hated it. Who knew? Not me anyway, and no one else seemed to care either.

STOP

Before doing any large scale of work, you need to stop. Never jump straight into the code, you will more often than not regret it in the long run. Unless your piece of work is for something short lived then you need to plan. I should have done an analysis of what was wrong with the existing app. The cost of fixing it, the timescales required to fix it, the risks associated with fixing it etc. On the other hand I should have done the same, but the risks etc for creating it from scratch using a new framework suitable for the requirements of the client.

With the existing, but broken app being considered a prototype and a stepping stone to making a better application, along with the raw data on risks/benefits etc given and shown to management (not just direct manager), then it would have been much more likely to have been successful when I suggested it needed to be redone.

Even on the basic needs that the users machine and monitors were so outdated, they needed a different solution. It would have cost the client a heck of a lot less to upgrade their PC’s and monitors, but that wasn’t an option. Why I’ll never know. Well I do know – but the reason made no sense, not a bit of it.

Key learnings

Ownership/Trust – I should have trusted in my ability to re-write from scratch the app to make it better. I knew I could, but when it came down to it I still didn’t have the belief to force my opinion to re-write it. Documenting the risks/benefits would have given that mental safety net, as carrying on with the old one wasn’t a viable solution at all.

Know your customer/client – Learn what the client has and what they use. Far too often I hear, it works on my machine. It needs to work on the clients machine.

Stop – Think about why you are doing something, what’s being planned, where is the future of the code going to be.

Life Sucks! Notes To My Younger Self.

Life Sucks! Notes To My Younger Self.

As a developer, especially as a junior developer things can take a turn for the worse and you probably won’t have seen the blinding obvious coming right at you… Life can seem to suck at times, but like everything, it depends what you take from it.

This is part 2 from the React JS/Finland talk that I had the opportunity to give on the 14th Sep 2022 (https://youtu.be/zE8PDM_7xoQ?t=11416). This sections goes to show that even though I had what seemed like the ideal first job, it really wasn’t.

My first role was to help with the creation of this game! 😎

First Job

So after a year of job hunting, I got my first role in a AAA games company. Exciting – yes, very much so. A large company with 3 separate games being made (in conjunction with other studios). Each team had around 30+ people in it. The place had it’s own arcade booths, we got fresh fruit every day, people would play online games during lunch. Variety of the latest consoles laying around the office. What wasn’t their to like!

Downhill

So in the door and on the whole, people were busy, very busy. They had headphones on and coding away – in the zone so to speak. I recall being shown a list of literally 1000’s of bugs that needed fixing before the game was suitable for release. Of course in those days, you couldn’t update over the internet. What shipped on the disk was what shipped into peoples homes and of course if the game didn’t make the Christmas deadlines for being created that was not a good thing!

So with everyone being so busy, support was next to nothing. I only recall a single meeting with management to see how things were going, and that was early enough for me not to really give a good opinion in it. Recall many times scratching my head, wondering what was happening with the code, but everyone had headphones on and ‘in the zone’. As a junior, certainly didn’t feel like I could/should be interrupting the other dev’s to ask ‘silly’ questions. None of the senior dev’s would ask how I was going, or if I required assistance.

Could it get worse?

Yes it could! Due to the main team being in the states (we were based in Scotland) and infrastructure issues (20 years ago, transferring 2-3GB of data nightly was problematic at best) the whole team had to fly out to the states in order to complete the game on time. But I was about to get married, so I couldn’t go…

So now I was working US hours, essentially on my own in a mainly empty office. But I was getting free food/dinner as I was working late, and I was still doing some dev work, just nothing really meaningful. Trying being a jnr in a code base many 1000’s times bigger than anything your experienced from university, it’s not documented in any way, PR’s and unit tests were not a thing, even source control was very sketchy.

Coasting

But did I see any of the above as an issue – NOPE! Why not, well I just didn’t have the experience. I thought once the team get back from the States, I’d get properly into the code, be able to ask questions, do some learning etc.

I knew things weren’t great, but at the back of my head, I was sure it would pan out and be fine!

Did that happen, nope… Redundant soon after Christmas 😒

Hard lessons

Looking back it was clear from the start that the company wasn’t interested in me. I was a number to allow them to get what they wanted. A decent company that hires a junior dev will never just leave them to flounder on their own. They will mentor and guide them. The simple fact that there was zero support was a simple and clear warning. So if this is your experience right now or at some point in the future – what to do?

You need to invest in yourself. They will not care, so you have to make the time to pull apart the code base. Dig into it framework, put tests around items and generally if anything looks different/new learn why. This is hard, but take it in small chunks. Create a small app and slowly copy parts of the larger code base into it to get them working in a context you know and control.

Managers

Always support a junior developer. If you happen to be a workplace that loves music to code (most of us do). Then make sure that you take off those headphones to ask how it’s going. Don’t just accept a – ‘yeah all good’ response. It’s easy to do so, as you’re busy and if they say they’re fine, then great, you can carry on with your important jobs… Make sure that they are good. Junior dev’s are the people that come with energy, enthusiasm, new ideas etc, you need to channel that into something meaningful.

As an aside, over the years I feel that the gain you think you get from being in the zone with headphones on and banging out code – is actually a downside. Yes being focused is great, but often being out of reach regularly from your fellow developers is not good. One place I worked got the balance right, but that will be explained in part 4 of my talk review.

Junior devs

You can’t learn effectively in isolation. You need a team around you, you need a variety of experience. If that isn’t there then it’s time to look for an exit strategy. Be preparing your CV, go looking to see who’s looking to hire etc. Get preparing small demos, learn the usual interview questions as so on.

At the same time you should and need to be more vocal in asking questions. Never stop asking questions, don’t keep asking the same questions, but do keep asking questions. It’s not only good for you to learn, but at the same time those you’re asking the question off will also learn. If something isn’t clear – is it a good bit of code? If it isn’t, then perhaps it needs refactored. Sometimes it’s good to discuss an area of code that hasn’t been touched in years as you’ll have forgotten it’s purpose. Either way it’s not a one way street when you ask questions. So as a senior developer, I welcome questions.

Landing your first job! Notes to my younger self.

Landing your first job! Notes to my younger self.

First ever conference talk complete! Something that was well worth the effort to create. Anyway I thought it would be great to put down my talk into words. I’ll put a post up per topic that was covered during my talk. I’ve yet to listen to my own talk, although I know I rattled through it faster than I wanted πŸ™‚ So here I’ll expand what was in my head as well as some of the areas that I didn’t have enough time to delve into.

For those that didn’t attend or watch the React JS Finland conference (https://youtu.be/zE8PDM_7xoQ?t=11399) – it was great. Such a warm and welcoming bunch of people. And some of the people there are just amazing and super friendly. Helsinki may not seem like place to go – but I highly recommend it. The developer community feels like it’s really big and cooperative.

Getting started

This was my first point and the main jist of it is – without work experience how to do you get that experience so that someone hires you? This is something I didn’t do before I got a job, although over my early years in various jobs, I did things like this – which seriously helped me get more advanced roles.

Perhaps out of this section, the one thing I didn’t have time to mention was that I’ve seen over the years, that those who while at university (or similar) are able to get some kind of internship or summer time work experience (even if unpaid/helping charities etc) developing software. They are much more likely to be hired first.

While this perhaps seems a little unfair, as many can’t afford to work for free or next to nothing, life isn’t fair. Which is why you need to put the effort into the next part if you didn’t or couldn’t get work experience while learning at the same time.

View from the tower in the conference centre.

Motivation & Ideas

Two key things will hold you back, motivation and a good idea. You can look online and I’ve seen so many to-do lists, for example. While these are just fine for a tutorial, they aren’t the kind of thing that will excite you to keep programming for hours and days on end.

Too that end you’ll need to find yourself a good idea. So where to get that from? Well, in the past picking an item from one of your hobbies or interests. You will no doubt find that someone else has created something else that’s very similar, but don’t let that put you off. As you develop YOUR idea, you will end up adding your own twist to it.

Mental rewards

Whiteboard, sticky notes and pens. Why? Some people in the software world don’t do this (not even electronically). These people have been developing for a good while and when you interview them you can tell that what they do isn’t structured/well planned as it could be. Yes it’s not as simple as get a whiteboard, but it’s a mindset and the Jira/whiteboard board needs to be part of it. So what should you do?

Well, you have your idea/subject. It needs broken down into a few big tasks ‘epics’, then each of those needs to be broken down into individual units/stories. Those need prioritised. This is your first business take away. You will learn to prioritise them and now you have a concrete example to talk about in an interview (that bit is super important). When you do work on your tasks, work on them one by one. Only ever do one at a time! If you complete a story and something comes out of it, maybe just a ‘tiny’ task, you think I’ll just stick it onto this task. DON’T! Create a new task and do it right after, but don’t join the two together.

Buy yourself a physical whiteboard

The whiteboard

This is where the whiteboard/sticky notes come in. Put up enough to complete an actionable item/epic. Some of these many be learning tasks (spikes) as well as actual coding tasks. As an example you may need to investigate bundling/build tools. Task 1 could be to find out the top 3-5 tools available. You can have new tasks per tool. You’d create a ticket per tool and that will involve doing a small demo per tool. Finally you’d have a ticket to evaluate each one against the others. So a simple find a suitable build/bundler for your idea may well have 7 unique tasks/stories.

The act of progressing a ticket from ready for work, all the way along to done/released is a concept that although simple will trigger those mental rewards. You’ve done something! Great! The smaller the task the more tasks you can get done. There is nothing more demoralising that being stuck on the same task for days. If this is being done during a few spare hours, the smaller you can make the tasks the more likely you are to complete them. A large task requires much more time to get the task into your head. Sometimes by the time it’s in your head, and you’re flowing, the time is up. Keep them small.

There is a whole heap of knowledge out there on Agile (for example https://www.atatus.com/glossary/agile-methodology/), this post isn’t to teach you that, but to highlight doing it is very rewarding.

Takeaways

  • Get an idea for a project from your hobbies/interests.
  • Create tasks that are as small as you can make them.
  • Only ever do one task at a time.
  • Prioritise tasks to get something out/completed.
  • Think about why you are doing something. As in why is task A more important than task B.
  • You will be learning new techniques, think about how you implement such learnings, how to enhance your learnings.
  • Think about your flow, how to you improve it. Did you do anything that suited your style of code/learning.

Coding is as much about the process of writing code, as it about writing the actual code. Can you describe that?

ExifTool, moving from cmd line to Powershell

ExifTool, moving from cmd line to Powershell

Previously I highlighted the issue in using Exiftool and getting it to add GPS data to a photo. Now that that issue is resolved (see here) we move on to getting it to work within Powershell.

Getting it to work within Powershell is important for myself as I’m using that to copy photos from my SD card onto my machine much much faster than the Sony software can (see here). So why not reuse this script and as well as copying it across, I can add GPS data and also copy it across.

The Command

exiftool -GPSLatitude*=56.9359839838 -GPSLongitude*=-4.4651045874 DSC00320.JPG  

Above we have a working command. Powershell can call executables in various ways as can be seen here.

Option 1 – The & operator

This failed as the parameters for exiftool require to be more than a single string, which is what the & does. If you look at the link to the MS article I tried all the & options it suggests and nothing was close to working.

Option 2 – Direct call

Direct call requires to add .\ if the item isn’t in the PATH or some other means to grab the correct folder location. I had the exiftool in the same folder as the script so .\ did me just fine.

Cut to the chase…

There is an issue with powershell, and the - cause it issues. See this issue listed here

This meant on my first pass I had to split it into 2 requests to correctly update the GPS.

$combLat = '-GPSLatitude*=' + $point.lat
$combLon = '-GPSLongitude*=' + $point.lon

.\exiftool.exe $combLat $f.FullName '-overwrite_original_in_place'
.\exiftool.exe $combLon $f.FullName '-overwrite_original_in_place'

Now with the use of backticks I can do the following

.\exiftool.exe `-GPSLatitude*=$point.lat `-GPSLongitude*=$point.lon $f.FullName -overwrite_original_in_place

Phew…

That tiny piece of code above (change to backtick) took so long and with a combination of human error and not knowing the order to use a ` or a ‘ or a ” or even a range of them required fair bit of trial and error.

It’s on Github

If you wish to view the full code, grab it, run it, etc then head over to this link.
I’ve seriously cut back this post as so many things I’ve learnt, but the easiest way for you pick up from my learning is to just view the code. A picture may well speak a 1000 words. But a 100 lines of code is so much better than 10,000 words in a blog πŸ˜‰

https://github.com/delp–/GPS_Sony_Photo_data_merger

ExifTool, Powershell, Cmd line – adding GPS

ExifTool, Powershell, Cmd line – adding GPS

So you’ve got some photos, those photos do not have any location data. You have some data and you wish to automate adding that data to your photos.
The concept is simple, yet there are numerous hurdles to overcome.

I’ll highlight some of the issues I hit along my journey so you do not have to. See previous post which is another hurdle I had to overcome.

It works on the command line

First step is to get the Exiftool from here if you wish to follow along. This does the actual adding, you just need to know what and how to call it.

Next up, is that you are going to test it working on a single image and most of the examples I was reading involved the command line. So off I went.

This is the XML node containing the data

<trkpt lat="56.9359839838" lon="-4.4651045874">
    <ele>30.42</ele>
    <time>2021-03-30T20:09:56Z</time>
</trkpt> 

So you pull out the latitude and longitude and come up with the following:

exiftool -exif:gpslongitude=-4.4651045874 -exif:gpslatitude=56.9359839838 -GPSAltitude=30.42 DSC00320.JPG

Important to note that I’ve copied the exiftool.exe into the same folder as the image and the above command is run from that same folder. Means you don’t have to worry about paths. Makes it simple for initial start up.

So I run the above, it works – sort of! No errors, I take the image, drop it into either Flickr or a free app that shows the location the photo has been tagged with. Hmmmm, it’s not right. That location is in Scotland, but the map is showing it as somewhere in the middle of the North Sea!

Reason it failed is that GPSLatitude/GPSLongitude needs a reference to it’s place in the world, as in which quadrant it’s in. So that would be N/S/E/W. But I do not have that data. In my case the data is negative, so that means it’s West and not East. But this command only takes the number as if it was positive! It completely ignores that fact I gave it a negative value!

Next option

So exif failed. Googling, I can see that if I use the xmp commands, they can handle negative values, as in it does not require the reference values of N/S/E/W

exiftool -xmp:gpslongitude=-4.4651045874 -xmp:gpslatitude=56.9359839838 -GPSAltitude=30.42 DSC00320.JPG 

Again no errors. I load the file into GeoSetter and it shows me exactly where I expect the photos location to be. Wonderful. It worked then. NOPE!!!!

I load the exact same file into Flickr, and yet again, the same as the first command it places the photo’s location into the North Sea! πŸ˜• Trying this in various ways and same thing each time. Works in GeoSetter, but not in Flickr. What on earth is going on???

Debugging/Comparing tags

Firstly I went to look at the GPS location while inside of GeoSetter and if I tried to change it there, it gave me the following warning.

After setting this in GeoSetter, the file with no changes to GPS location then worked in Flickr!

So now that it worked in both GeoSetter and Flickr I presumed that the reason it failed initially in Flickr was due to some timestamp/date issues. Perhaps Flickr being a bit more professional than a free app, it respected the rules in a stricter fashion. WRONG, the reason it failed was not down to the timestamp! On adding the timestamp, Geosetter also updated the geotags which made it work in Flickr.

Had I known this command at the start which outputs exif tags in groups that would have help hugely in debugging the situation.

exiftool -a -G1 -s E:\CameraAddGPS\TestFolder\before\photos\DSC00939.JPG

Correct command to insert GPS with only lat/lon

exiftool -GPSLatitude*=56.9359839838 -GPSLongitude*=-4.4651045874 -GPSAltitude*=30.42 DSC00320.JPG 

You can see that the parameters include a wildcard and what that allows is to only give the lat/lon and it will then work out the N/E/S/W based on the -ve or +ve values given. πŸ‘Œ

Now that works in both Flickr and Geosetter. Brilliant 😍
Next up is to figure out how to do so from within Powershell. (see my next post, coming soon)

Additional info

Just a FYI, if you run the above command to output the grouped data on the files, the below is what you see. The text on the left if from the data of the file that works in all places (Flickr and Geosetter), and the text on the right is from the file that only worked in Geosetter.

You can see that both have the correct data, but the one on the left (which works in all places) has the data inside the main GPS tagged group, whereas the one on the right has some different Composite data and XMP-exif data which isn’t standard enough for Flickr to pick up.

Powershell – What on earth are you returning?

Powershell – What on earth are you returning?

I’ve finished a ‘minor’ project where I extended my previous Powershell script seen here – https://www.kennethsutherland.com/shell/copying-files-using-powershell-sd-card-to-pc/

What’s new and what did I solve?

I’ve got a Sony RX10-IV and one of its drawbacks is that is doesn’t have a GPS unit inside it. This is a major bummer as far as I’m concerned. So what to do? I searched, read various guides and yet nothing simple jumped out to solve my issue. Ideally I wanted a device to attach to it’s hot-shoe that when I took photos it would write GPS data to the photo. There is nothing, not a single thing that is suitable for the Sony camera.

Before anyone comments, why not use Sony’s phone software to sync up – it’s a huge pile of πŸ’©πŸ’©πŸ’© Not getting into why, it just is! PlayMemories and Imaging Edge Mobile are both virtual paperweights. Ready to be binned on my phone now.

So I happen to have an eTrex 30 for when I go hillwalking and as soon as you turn it on, it starts recording it’s location and time. This is stored in a *.gpx file. Sorted! All I need to do is to merge the gpx file with my photos. Sounds simple. Actually nope, took me far to long…

Functions that return way more than you expect!

So you can create functions in Powershell – good.

You create the function flow, and in order to make sure it’s working as expected you put in some echo commands. Great so far.

NO, NO, NO – do not use echo commands inside a method that you are going to return something. It completely MESSES it up!

Some code ->

Look at the below screenshot while debugging some code πŸ‘‡

If you look at the above image πŸ‘† you can see that the method getSelectedPoint returns the selected point. But it also has a echo.
Inside the debug tooltip you can see that the returned object is more than the point, it’s also got the echo statement!
WTF Powershell! Never seen a language return more than what’s specified to return.

Now check out the same but without the echo

Look at the same debug tooltip when the echo isn’t there. That returned point is just a point.

WARNING – I don’t know Powershell

Must add this warning, as I’m sure those with experience will call out the reasons for the above and no doubt issues with my code. But as a novice in powershell I’m highlighting the issues I stumbled on to get the end result. This took to long to figure out. HTH.

If you’ve got this far and want to know more then head along to this – About Return This is a MS guide about powershell and what it returns. Essentially if you want a standard programming style return, then you can use a class.

Spread and Exclude

Spread and Exclude

The other day I wanted to log out some values of a large object. But one particular property was massive and not needed. So how could I pass all of the original object minus that one property that was going to get in the way?
As in how to exclude a single property from a object.

Solution:

It’s a very simple and I think elegant solution. so here it is.

Using the spread operator you can list the specific items to copy first (these will then be excluded) then use the … rest operator to copy the remaining items.

That’s it! It telling your code what to specifically copy first, those will then not be included when you use the rest operator. Brilliant!

const bigObject = {a: 1, b: 2, c: 3, d: 4, e: 5, f: 6};
const { a, ...copiedAndExcluded } = bigObject;
// copiedAndExcluded is now the same as bigObject, expect it doesn't have the property 'a'.

Also in my case the bigObject was a property of something else, which at times may have been undefined. This will cause an error, as you can’t spread undefined.

const parentObject = { a: 1, bigObject = undefined, c: 3, d: 4};
const { a, ...copiedAndExcluded } = parentObject.bigObject || {};
// 'a' and 'copiedAndExcluded' will be undefined

The `|| {}` will make sure that the code doesn’t try to spread the value of undefined (as that will throw an error). You can however spread an empty object. The values that come out will be undefined, which in my case was just fine.

Copying files using Powershell (SD card to PC)

Copying files using Powershell (SD card to PC)

I’ve got myself a new camera – Yeah 😎
It doesn’t come with any software to copy images etc over. Boo…
It does say you can manually copy from SD to your machine, but frankly that sucks. I expect software to create things like a nice folder structure automatically. I’m not going to copy, create folder(s), paste multiple times each time I take some photos.

It’s a Sony RX10 IV, so it’s not a cheap camera, and after a hunt online there are offerings that Sony say to use, as in the Capture One software. I grabbed that, gave it a go. Had just over a 1000 images on the SD card that I’d taken (the card has a transfer speed of 250 MB/s) so it’s no slouch!

Fire up Capture One (which has a very annoying register interface) and started to transfer the images. All seemed good.

SLOW!!!

After a short time the Capture One software was reporting it would take another 3 hours and 20 minutes to transfer the photos!
Over 3 hours!
What on earth was it doing?
So I canceled and closed it. Reverted back to my Nikon software (used for older camera), after all, all I wished to do was to copy the photos from one location to another, and at the same time get the tool to create a reasonable folder structure at the same time.

Guess how long that took? I didn’t time it, but it was less than a minute, I’d approximate around 30 seconds.

So the issue wasn’t the card or my computer – it was Capture One.

Options

Looked online for other Sony recommendations, but all I could find was others complaining about the Capture One software – from years ago. Could I use the Nikon software? No I couldn’t, it worked fine for the jpg’s, but it was the Sony RAW files that caused an issue. They have a different file extension than what Nikon use. So their tool doesn’t see those files.

Thought, I could write a wee script to change the file extension of the Sony RAW files to the same type as Nikon. That would be simple.
Then I’m like why don’t I just write a bat file to copy across and create the desired folder structure.

Powershell – not bat

Firstly task is the script wants to look at each file, see when it was created. Use that data to then create a folder structure. That, isn’t actually a simple task with bat files, and the recommendation is to use Powershell. πŸ™‚

First powershell script

So I’ve never created a powershell script before, but as you’ll see in the following code it’s dead easy and very simple to work with.

# SD Card location
$inputDir = "G:\"
# Where to copy file to
$outputDir = "D:\myPhotos\"
# get all files inside all folders and sub folders
$files = Get-ChildItem $inputDir -file -Recurse

# Create reusable func for changing dir based on type
function copyFileOfType($file, $type) {
    # find when it was created
    $dateCreated = $file.CreationTime
    # Build up a path to where the file should be copied to (e.g. 1_2_Jan) use numbers for ordering and inc month name to make reading easier.
    $folderName = $outputDir + $dateCreated.Year + "\" + $dateCreated.Month + "_" + $dateCreated.Day + "_" + (Get-Culture).DateTimeFormat.GetAbbreviatedMonthName($dateCreated.Month) + "\" + $type + "\"
	
    # Check if the folder exists, if it doesn't create it
    if (-not (Test-Path $folderName)) { 
        new-item $folderName -itemtype directory        
    }
    # build up the full path inc filename
    $filePath = $folderName + $fileName
    # If it's not already copied, copy it
    if (-not (Test-Path $filePath)) { 
        Copy-Item $file.FullName -Destination $filePath        
    }
}

foreach ($f in $files) { 
    # get the files name  
    $fileName = $f.Name
    if ( [IO.Path]::GetExtension($fileName) -eq '.jpg' ) {
        copyFileOfType -file $f -type "photos"
    }
    elseif ( [IO.Path]::GetExtension($fileName) -eq '.arw') {
        copyFileOfType -file $f -type "raw"
    }
    elseif ( [IO.Path]::GetExtension($fileName) -eq '.mp4') {
        copyFileOfType -file $f -type "movies"
    }
    else {
        #Do nothing
    }    
}

How to run?

Three steps are needed.

  • Copy the above code and paste it into a ps1 file. So give it a name such as transferFiles.ps1 (really doesn’t matter)
  • On the first instance, open Powershell as Admin
    • run this Set-ExecutionPolicy RemoteSigned
    • this lets you run you script locally, if you don’t do this and try to run any powershell script that you create, it will give an error.
  • Run the script. Either right click and ‘run with Powershell’ or call it from the Powershell cmd.

Explanation – change it?

I’ve included a load of comments, so hopefully it should be clear. Essentially if you want to use it yourself, you will wish to change the

  • $inputDir To wherever you original images are.
  • $outputDir Where you’re existing photo root folder is.
  • $folderName is what I’ve used to define the created folder structure. Feel free to change to your personalisation.
  • The RAW file type extension of Sony is arw. Your type may be different. I split the different file types into different folders. That’s just my preference. Do what you wish.

Result

The above shows what the output is. Simple folder with year, then combined month and day and the named month. The reason I do this is for ordering via the name. If the date was first then you’d get the 1st of each month together, so the whole list wouldn’t be in chronological order when in alphabetical order.

Improvements

  • I could make this happen automatically when the card is inserted into my machine.
  • I could delete the files once it’s copied (and added verification that they’d been copied!).
  • Could add in duplicate coping to my back up NAS drive

Sure with this being my first go at a powershell script there are ‘wrong’ items, but hopefully it will give someone an idea of what to do. Hopefully it works at speed on 1000’s of files as well! I’ve only tested it on a few files right now. Gives me an excuse to go out and fill the camera SD card with a load of photos 🀣

Cycles of Life – A Cautionary Tale!

Cycles of Life – A Cautionary Tale!

β€œThose who do not learn from history are doomed to repeat it.” (George Santayana)

Having been around the software world for more than a few years you see the same cycles time and again. That rather old phrase of “those who do not learn from history are doomed to repeat it.” comes to mind.
Teams grow and shrink and all going well grow again, prepare for it, so you can mange it.

More appropriately this should be called –
Cycles of Life a Software Team – A Cautionary Tale!

Software and the teams around them are on the whole very versatile and nothing is really doomed. It will probably just slow (hugely) you down or cost someone a shed load of cash to sort out πŸ˜›

We’re entering a new year, all going well your team is looking forward to growing, so keep the following in mind!
When you know what to look for then hopefully you can do something about it.

Team growth

  • How big is your overall team?
  • How big are the individual teams within the overall team?
  • How quickly has the team grown?
  • How quickly do you want it to grow?

These questions are key to figure out what will happen. The following has happened on more than one occasion and in completely different teams.

How quickly have you grown?

You are an awesome start up, your idea is a massive hit with the investors, it’s time to grow, grow, grow!
BRILLIANT!

Here’s the thing, the software industry has a HUGE staff churn rate. Even the best companies will struggle to keep staff for a long time. You must be prepared for this.

Your team has grown from 10 to 30, maybe even 50+ in a very short period. The buzz around you encourages others to join. Nothing better than a greenfield project to work on.

Here’s the rub, all those new staff will be very likely to leave in very quick succession after 18-30 months, once the first goes. Sure some will stay on longer, but also some of the core team may also leave. Through no fault in the team or the project, but just to go experience something else. This is completely normal in software teams. Especially once a senior long standing member goes – look and plan immediately for other to leave.
There is virtually nothing you can do to stop the flow, just accept it will happen and deal with it.

So what!

I’ve hired before, I can hire again. It’s all fine. It’s just a coupe of people.
If it’s not already clear, the short life span for a developer in one place, means if you hire lots in quick succession, they will leave in the same fashion. Unless you are still actively growing, you’re in for a bump!

It’s the wave effect

This point in time, you are no longer a start up, you no longer have a greenfield project, you no longer have the ability to just hire another dozen developers.
Also notice and look for a bit of a build up, like a wave really. Their is a feeling of general minor grumblings. Nothing major, but then once someone goes, curiosity and a question of ‘is it greener over there’ without something to cancel out these thoughts means that inertia to leaving is broken.

Managers, why?

I have to admit failing to understand this point, but senior management never wish to replace a member of staff leaving right away. They seem to like the thoughts of, well it’s only 1 person out of 40. We’ll make do without them.

Sure enough, the team does, it’s only 1 person. But that wave is building up in the rest of the team. Then the 2nd, 3rd go. By which time those leaving encourage others to go, maybe even taking there mates with them.

Know your market!

Depending on how quickly you can hire. Every market is different and you need to know where you are.

  • Maybe your company is in a software hub of a city and hiring isn’t an issue.
  • Maybe your company is in a software hub, but part of a structure that makes hiring a labourious process
  • Maybe your company is in a remote area as far as getting a pool of developers is concerned

Essentially how quickly can you get people back in that door. Are you ready to take staff out of their day jobs, plough through CV’s, then do a fair amount of hand holding to get the new staff up to speed?

Even in the best situation where your company has a low bureaucratic process for hiring, you develop in a good cultural hub/city and you have the finances to pay a potentially larger staff bill (generally a case of the best way to get a raise is to move on). You will have to take team members of active duty for transitioning in new staff.

Recommendations

  • Plan for people leaving – it will happen.
    If you quickly back fill a space, it will reduce the desire of others to look around. Anything fresh is a good thing, even a new face aids.
  • When you know it will take a 3 month minimum period to hire someone, never leave it for 3 or more people to leave. If you do then then even if you start hiring, those seeds of change will have sunk in already.
    Really if 3 people have left and you’ve not started looking, then you’re guaranteed that others will be on the way out (and in quick succession too, esp if they we’re all hired within a similar time frame).
  • Do not ignore unhappy staff – they’ll just leave quicker.
    You may well, and quite possibly completely disagree with your staff members reasons for being unhappy, but it’s often an open market.
    Do you want them to stay or not? If not, plan for them to go. If they stay, great, if not, then you are ready.
  • Ideally always look to the weaker area in the team. Have a range of CV’s coming in and keep the top ones for reference later. If you have potentials ready to roll, then anything you can do to reduce a gap is crucial.
  • Most importantly it can take a number of months to gain enough domain knowledge. The longer you leave it to re-hire folk,this will increase the risk of crucial knowledge being lost, which will likely lead to a major issue that could be solved quickly, but only if the domain knowledge was retained.

Finally…

Stop focusing on just the technical. You need to be a team, personally as much as just work colleagues. This is SUPER hard, sometimes you want nothing more than to just do the work and go, but…

  • The grass isn’t always greener on the other side! So keeping your staff happy, when they go, they will spread the good knowledge and practices of your team.
    This can even aid in then getting new staff in, but it can also mean that staff can and will come back! Having been in many different companies, the programming world can feel like small circles at times. Just because someone left doesn’t mean they won’t wish to come back.