The scenario I want to talk about is using temporary tables to materialise a set of data, ahead of using it in a separate query a moment later. That’s essentially what Jeff Taylor is asking us to comment on.

Temporary tables are fine… really. I use them often.

If I’m writing queries in SSMS, to do some quick debugging of a situation, then I’ll often drop the results into a temporary table.

If I’m dealing with remote queries across a linked server, I’ll often use openquery and make a local version of the result set.

If I’m grabbing a tiny subset of data from large tables, scanning to look for examples of particular scenarios, I might use NOLOCK and temp tables, before re-testing those specific cases without NOLOCK. More on this later (because I know a bunch of you screamed and unfriended me when you read that there’s a scenario I used NOLOCK for).

And of course, I might use temporary tables to look at the current wait stats before waiting a moment and re-querying – because that gap in the query is what makes the time-based logic of wait stats work.

The problem with temp tables (or cursors or scalar functions to some degree) is that people use them as procedural crutches to ensure the correctness of their queries, and never refactor the code to make it less procedural. Imagine you are really confident in a piece of logic that you created using temp tables or a cursor, but you know that you really ought to use more set-based methods instead. Do I mind that you used procedural code to work out what you wanted? No – not at all. But how about you use that code in your testing harness, and come up with a better way. Your procedural code can be used to make sure your logic is correct, but maybe stop running it in production, yeah? Particularly once you have the skills to make something that performs better.

And as much as it frustrates me, there are times when procedural code is just better. Expertise in your domain should mean knowing when some of those ‘rules’ (about procedural v set-based) can be broken.

Let’s quickly consider that scenario with NOLOCK, for example.

Imagine you have a known application bug where sometimes the OrderTotal of an Order becomes different to the sum of the OrderLineAmount values. Shouldn’t happen. But it can. Arguably, Order doesn’t need an OrderTotal value, but for the sake of performance and various other things, it does.

Sure, you can explore options with triggers to try to spot it, but I’d be very reluctant to create a trigger on an existing system. One very real option is to run some hefty scanning queries again the tables involved to check the data. I’m fine about using NOLOCK for this kind of thing, because I don’t need it to be completely accurate. NOLOCK can (and will) give inaccurate results, so I drop the results of my scan into a temporary table (probably a table variable actually), and then run the query again without NOLOCK, filtered to the list I came up with, and now able to seek into just those potentially-problematic orders. To see if any of those rows that popped up were just because of data movement causing NOLOCK inaccuracy, and all while minimising the blocking. Is it perfect? No. Am I doing things that would make someone cringe? Sure. My choice though and it meets the goal I’m after.

When I’m running a query that involves a linked server, I feel like I’m even more at the mercy of the Query Optimizer than usual. I’m always somewhat at the mercy of the QO to be fair, but when linked servers are involved, I’m much less able to provide the QO with metadata, based on limited information about “what’s on the other side”. If the stats are looking like I might get a bad plan (with the potential for it to be way worse than a normal bad plan, and potentially impacting two servers), then temporary tables can be my friend. I can work out what data I’d from the remote server overall, fetch that in a single call (dropping it into a temporary table), and then using my new local copy of the data in my main query. Procedural code can legitimately help when you run the risk of having a bad plan across two servers.

Back in the day, temporary tables were often used as a way of breaking down queries to make them more understandable. And there were even times that the QO would prefer to optimise multiple smaller queries that a single large one, making temp tables a reason for performance. And as there could be stats on the temporary table, sometimes it would just work better.

Still today this is a very real tuning method, but I feel like the QO has improved a lot and there are advantages to be lost by materialising intermediate results. When you see a plan operator pushing out 10k rows of data, it’s better not to just let the engine use those rows, rather than drop them onto disk and pick them up again. Then again, if that same branch appears multiple times, then spooling them could be useful. And a temporary table is basically rolling your own table spool. It’s definitely a real tuning method!

I will always advocate for the power of the QO, and for its ability to see shortcuts to avoid work. But sometimes you really have to give a nod to query writers for creating stuff that the QO is going to struggle with, and either introduce or rip out temporary tables to get the behaviour you want.

And ultimately, I’d rather a query be easy to understand and perform well. I’m not going to rule out temporary tables if I can achieve both of these.

It’s all well and good to be a purist, but pragmatism usually wins.

@robfarley.com@bluesky (previously @rob_farley@twitter)

The post Temp tables – a blessing or a curse? appeared first on LobsterPot Solutions.

For the most part, upgrades to SQL Server 2025 go pretty smoothly. I’d even say upgrades to other versions of SQL Server have gone pretty smoothly too, although plenty of people suffered from performance pain from the change in cardinality estimation model when upgrading to SQL 2014 (well, the compatibility level 120). The ‘simple’ fix for that one was to put the compatibility level back to 110, and then figure out what was causing the pain. But with SQL Server 2025, there’s another pain point which isn’t quite so easy to get around.

Regular readers will know that I don’t like to write about real situations that customers have been through. It’s all well and good to say that I fixed the performance of a query by adding an index, or by changing the data type of a column, but I’m always reluctant to mention particular products or customers by name.

But this one has already been mentioned, including calling me out by name, so I’m fine.

David Musgrave is a Dynamics MVP in Perth (Adelaide’s nearest neighbour to the west, only 2000km away) who reaches out from time to time. A couple of months I got a call from Arthur Achilleos saying David had recommended me to figure out why a machine that had been upgraded from SQL Server 2019 to SQL Server 2025 had seen its CPU metrics skyrocket. Queries were struggling to run, and people were scrambling to figure out if rolling back to SQL Server 2019 was an option. The application was Dynamics GP.

SQL Server 2025 introduces a handful of new features. Intelligent Query Processing, for example, has seen a number of changes such as Parameter Sensitivity Optimization and Cardinality Feedback. A quick look at the queries that were the most expensive showed a lot of queries that had over a hundred parameters. My initial thoughts were that this could be painful if the engine is trying to assess the parameter sensitivity across all of that. But that wasn’t it. Not entirely.

Connecting via SSMS wasn’t much of a problem, and queries seemed to run well enough there.

We looked at the wait stats, and some of the usual suspects around parallelism were around, as well as some SQL Operating System ones (SOS_SCHEDULER_YIELD, for example). But there was one wait type that I hadn’t seen much of before. It was appearing third or fourth in the list of largest waits, and only occasionally jumping to the top, even when the long list of “ignorable” wait types were ignored (as per SQLskills’ list). It was the PREEMPTIVE_OS_CRYPTOPS wait type. I’ve linked to SQLskills’ post about the type, because they make it clear that this is typically more of a Windows problem than a SQL one (plus, SQLskills is where you should always go for information about wait types – it’s a great resource). Their page suggests to “have your infrastructure team investigate performance issues with servers that provide cryptographic services to the domain (for instance, the server for an Enterprise Key Management system)”. And while that’s really good advice, it wasn’t what was going on here.

This problem is actually inside SQL Server 2025, and what’s worse is that you probably won’t notice it in your test environment unless you’re testing at scale.

The issue is the way that SQL Server 2025 hashes the passwords for SQL Logins. Dynamics GP uses SQL Logins. Lots of older applications do. Essentially, the PBKDF2 algorithm improves on security by applying the SHA-512 hash 100,000 times. The idea is to slow down brute force attacks, but it will also slow down applications that re-authenticate a lot. Chatty ones. Connection pooling can help to a degree, but not as much as you might like. Best is to use a Windows login instead – the trusted account for your web service, or the user who’s running the application. It’s why SSMS might still run just fine – because SSMS probably connects as you. And because SSMS won’t be as chatty as your application.

Your test harness, making sure that your logic is correct in the new database environment, might not push enough new logins to notice. Maybe it reuses its connections really well. Either way, this is a problem that seems to avoid getting caught in test environments. Perfect.

Let me be clear… The fix is to use a Windows Login. But I do appreciate you might not be able to change your application.

You can’t just set the compatibility level back to SQL 2022 to make this problem go away, but there is a trace flag you can use. A couple of months ago, it was definitely undocumented. I’ve heard it’s now documented, but I’m not completely sure. I can’t see it described on anything official. So maybe check with Microsoft Support. (Even though when your CPU is high and your application is failing, you want a really urgent fix and don’t want to have to wait for Microsoft Support!)

But even the trace flag isn’t the whole fix.

You need to know – this trace flag was documented by Michael Howard for SQL Server 2022, for turning this behaviour ON. You can read it at Support for Iterated and Salted Hash Password Verifiers in SQL Server 2022 CU12 | Microsoft Community Hub. But in SQL Server 2025, the trace flag reverts you to the old one. You might want to be aware of that if you used the trace flag to become compliant with NIST SP 800-63b. Personally, I would’ve preferred a different trace flag for turning PBKDF2 off, that wasn’t the same one that turned it on before. The trace flag is 4671. There are a few blog posts about it, but not enough to convince me that it’s fully supported. If you turn it on in SQL 2022, it might make your application slow. It’s annoying.

The trace flag was in that last paragraph. But please keep reading for the full fix.

In SQL 2025, after you’ve applied the trace flag, each password hash will still be the PBKDF2 version until you reset it with ALTER LOGIN, so just applying the trace flag won’t fix the CPU right away. The login mechanism is smart, and checks to see which hash algorithm was used so that it can check it right way – either the fast method or the slow one. But in SQL 2025 without the trace flag, it’ll update a successful the hash to the PBKDF2 version for next time. So the trick is to turn on the trace flag, then reset all your passwords (let’s face it – probably only one), and only then will your performance return. Luckily, the logins will still work on either algorithm, but anyone logging in repeatedly with a PBKDF2 hash will hurt your CPU until you reset the password.

The clue for spotting which algorithm was used on each password is to look at the password_hash column in sys.sql_logins. The PBKDF2 ones start with 0x03, and the older/faster ones start with 0x02. Leave ‘sa’ disabled (always) and with a PBKDF2 hash, because that’s the one you really want to be more secure (same for all sysadmin ones), but any locked-down application logins could be 0x02 style, for the sake of your CPU.

And in case you’re curious, David’s post about it is here, including some instructions on configuring trace flags.

I really hope you’re reading this BEFORE you’ve experienced it. Otherwise, good luck.

@robfarley.com@bluesky (previously @rob_farley@twitter)

The post SQL 2025 showing crazy-high CPU appeared first on LobsterPot Solutions.

2004 was a different time for T-SQL. For example, we could still write joins using commas, putting the predicates in the WHERE clause. This still works today, but back then if we wanted to do an outer join, we would join using a special operator, such as “WHERE s.ProductID *= p.ProductID”. Code like this was deprecated in SQL Server 2005, and stopped working completely in SQL Server 2012.

So let’s start with that one. Hey Brent04, stop doing joins that way. Embrace the ON clause. It won’t make your code work any better, but it will stop it from breaking in a few years’ time. Plus, you can do more with it, having control about which join predicates are used for joining which tables. If you do find yourself wanting to use a comma, write out CROSS JOIN instead. I know it’s more typing, but you’ll make better choices.

But now that we’re talking about joins – start using aliases, and don’t just refer to the first table as ‘a’, the second as ‘b’, and so on. It doesn’t change the performance, but I will judge you. If you’re only using aliases when you refer to a table twice in a query, then I’m going to think you write your queries using Query Builder, and that’s something you should move away from (Query Builder will limit you, and can even make mistakes, so don’t use it – take the time to learn to write queries without it). Without aliases, at some point you’ll look back at your code, and when you see something like: “WHERE a.Col1 > 3 AND Val < 7”, you won’t know which table the “Val” column is in, and you’ll need to keep looking back at the FROM clause to see which table is ‘a’. I’ve seen code like “FROM dbo.Bank as a JOIN dbo.Account as b ON a.BankID = b.BankID”, and that’s going to trip up anyone.

Notice how I used schemas there? Yeah, you should specify the schemas. Don’t be lazy. We all do it from time to time, especially in databases where everything is in dbo – but at some point you’ll regret it. Plus, it’s slightly more work for the database engine if you don’t tell it the schema, as it has to check what your default schema is, and see if that object is in there. But more importantly, when your code runs as someone who has a different default schema, it might do something different.

Another pattern that will really make me cringe, Brent04, is when a procedure contains lots of UPDATE statements. I’m sure you know what I mean – this kind of thing:

UPDATE #WorkingTable SET Col2 = Col2 + 1 WHERE Col3 BETWEEN 5 and 10;
UPDATE #WorkingTable SET Col3 = 11 WHERE Col2 > 10;
UPDATE #WorkingTable SET Col2 = Col2 * 2 WHERE Col3 > 10;
–followed by dozens of similar lines

This code tends to have been created by people who have a process they need to follow. A recipe. And they can get it working, but it becomes huge and error-prone, and no one wants to debug it. When someone works out the logic isn’t right – they tack more statements onto the end.

Now, I have my ways of handling this. I start by using a lot of CROSS APPLY clauses to create each new version of the table, with CASE statements to replace the WHERE clauses. It probably looks even more ugly than the list of UPDATE statements, but hopefully does everything in one fell swoop, and after testing the results and tweaking until it’s correct, I can look at the expressions that get used in the plan to figure out what the finished result should be. Worst case, I leave it as the CASE-statement version, which is almost certainly quicker than the series of UPDATE statements, which might look something like this:

UPDATE v0 SET Col1 = v3.Col1, Col2 = v3.Col2, Col3 = v3.Col3
FROM #WorkingTable v0
CROSS APPLY (SELECT v0.Col1, CASE WHEN v0.Col3 BETWEEN 5 AND 10 THEN v0.Col2 + 1 ELSE v0.Col2 END as Col2, v0.Col3) v1
CROSS APPLY (SELECT v1.Col1, v1.Col2, CASE WHEN v1.Col2 > 10 THEN 11 ELSE v1.Col3 END as Col3) v2
CROSS APPLY (SELECT v2.Col1, CASE WHEN v2.Col3 > 10 THEN v2.Col2 * 2 ELSE v2.Col2 END as Col2, v2.Col3) v3
;

If you do this, make sure each CROSS APPLY clause can return exactly one row for each row in the base table (for example, use an aggregate if there’s a FROM clause, or OUTER APPLY and TOP (1), or don’t use a FROM clause and have no filter like in my example).

So, Brent04… you know I could keep writing for a while here. I’m going to suggest that you don’t beat yourself up from all the advice you’ll get from this T-SQL Tuesday event. Bad code exists, and it takes time to get rid of bad habits. Do what works for you. Understand that correctness is more important than performance, and that you need to be coding within your means. If someone (even me) has recommending a method you don’t understand, seek to understand it before you start using it. I see a lot of people who have tried to use patterns they’ve been told are good, but do it wrong and end up with an even bigger mess.

Good luck Brent04 – I guess I’ll see you in about 5 years. I think it was 2009 when we met in person.

@robfarley.com@bluesky (previously @rob_farley@twitter)

The post Signs of potentially-bad code appeared first on LobsterPot Solutions.

One of the things that Koen asks about is training people new to on-prem – things they wouldn’t necessarily know if they have only worked in the cloud. My mind turns to disk-related issues such as restoring from log files, and to corruption. There was a while when database corruption seemed to be quite commonplace – or at least a reason for people to get in touch with me to help. It’s definitely a lot less of an issue nowadays, even for on-prem environments, but the underlying concepts of disaster recovery are definitely something I think should be understood by SQL professionals.

I feel like every SQL professional should be able to argue their reasons for choosing a particular frequency of log backups, where those backups are going to, and what happens to those files. Is recovery tested regularly, including to a specified point-in-time? And can the data professionals explain what’s needed for that? Do they understand tail-log backups, and how to take one if the LDF file is all that can be salvaged (and do they even understand the importance of salvaging the LDF file)?

I like to think that junior DBAs are trained in this even if they never use on-prem environments. But I’ve definitely come across gaps.

Reliance on data is more and more significant for all organisations, but more and more of that data is in SaaS systems, where the data is stored on the servers of those separate systems. We don’t think about how this data is stored in these online systems, let alone how to back it up and what those companies do to make sure they can recover our data in case of disaster. We might extract the data for analysis, but if these systems become on-prem once more, running on local databases with local DBAs, understanding how to recover will be critical. 

I do wonder how many organisations would cope if they had to pull systems back from online systems. I have clients who only use systems that can guarantee the data stays in Australia, but this doesn’t tend to include “controlled by Australian organisations”. Considering news articles in 2025 (such as at https://www.forbes.com/sites/emmawoollacott/2025/07/22/microsoft-cant-keep-eu-data-safe-from-us-authorities/) reported that Microsoft could be forced to “snoop” on EU-hosted data – although they would push back on any such requests (https://broadbandbreakfast.com/microsoft-pledges-to-protect-european-operations-and-unveils-data-center-expansion/) – I wonder whether those concerns should be raised even higher. Is anyone asking the smaller SaaS companies what they would do if the US government asked them for sensitive information held in non-US data centres?

One thing is for certain – if tension increases between your country and the country of the organisation that has your data, then you will have an interesting time ahead.

@robfarley.com@bluesky (previously @rob_farley@twitter)

The post Forgotten skills that could become critical appeared first on LobsterPot Solutions.

I’ve used quite a few different methods over the years, because there are pros and cons of each, and what will work best for one situation might not work at all for another. For example – it’s rarely okay to make changes to an existing application, but sometimes it’s okay to replicate the data to a second database, and use that to detect changes. However, if your database doesn’t have primary keys, then some kinds of replication won’t work either…

In an ideal world, the database is controlled in-house, so that methods like Change Tracking, Change Data Capture, or the new-in-2025 Change Event Streaming (CES) can be used. Bespoke applications written by a local development team can often agree to avoiding the restrictions that these methods have, but it all depends on what’s needed.

For example, I really like CES. It’s an easily-configurable tool which will stream changes out to Azure Event Hubs using standard protocols like Apache Kafka or AMQP. The data appears in a CloudEvents message using JSON format, and includes the old values and the new values (how good is that!), so it’s easy to look in the event stream for messages that suit particular purposes but also dump the data into longer-term storage to let you go back to it. I wouldn’t want to have to search through a backlog of changes from years ago in JSON format, but it’s easy to push the changes into a table for the next ETL load (which can save me trying to use start/end times to pick up the changes for each ETL load, and hoping that I can rely on arrival times), and also store the data in a different format to suit a longer history of changes, at whatever grain I like. Tracking every row with triggers feels a little too close to the application in most cases, while using temporal tables physically adds columns (even if they’re ‘hidden’ columns that’s probably not a good thing) and locks the tables down a lot.

But every silver cloud has a dark lining… You know how I mentioned that local developers can often code around the restrictions here? Well that’s a thing.

At least at the moment (and I’m not saying that because I have inside knowledge – I don’t have any inside knowledge about this at all), there are some limitations.

Some are fine – for example, you can add or remove columns, and CES doesn’t care. If a new value appears in a column that wasn’t there when CES was first configured, that’s fine! The message going to Event Hubs will reference that new column (it’s just JSON after all), and hopefully your code handling the stream can handle that. Maybe have a column for “other column data”? But if you want to rename a column that’s in a CES-enabled table… sorry, that won’t work.

If you’re using partitions, CES isn’t for you. You might not notice it on day one, but when you go to use SWITCH, then you’ll get an error. Worth knowing. Oh, and you’re probably not using TRUNCATE TABLE on a table you’re tracking, but stranger things have existed.

But when we get to the column restrictions, there are some surprises. If you’re someone who uses geometry and geography, then great – I always felt like there should be more people using those data types. But json and xml are also not supported – and I do see a lot of xml columns. Like I said, if you’re wanting to use CES in a local environment, then maybe you can code around this? Or consider CDC, CT, temporal tables, or plain old replication, which all support those types because they don’t need the data to be serialised as much as CES.

For me, CES is very promising for detecting changes, and gives me a bunch of flexibility about how I do that. It’s very probably my preferred method for tracking history, but the limitations are still pretty annoying. Hopefully in vNext…?

@robfarley.com@bluesky (previously @rob_farley@twitter)

The post Detecting changes appeared first on LobsterPot Solutions.

Steve is hosting this month’s T-SQL Tuesday, and asks about what we’ve learned from conference sessions, things which impacted us and how we work. It’s an interesting topic for two reasons – firstly, I enjoy giving conference presentations, and secondly, they’re really not my preferred way of learning.

Don’t get me wrong – conferences are a great way to learn things for a lot of people. I just find that my brain isn’t wired for learning during sessions. If I’m at a conference, I’m probably learning more from conversations, and from hearing what people are saying outside the sessions they’re giving. But this is part of why I enjoy giving presentations. Let me explain, by way of describing some sessions where I definitely did learn something.

I’m not sure which things I learned in conference sessions per se, rather than at conferences in general. I remember an incredibly insightful session at an MVP Summit in 2023 where a few things about Fabric really clicked in ways they hadn’t before. It was so insightful that I grabbed a few others who hadn’t been there (Alex Whittles comes to mind) to join the rest of Bogdan Crivat’s session. But I can’t say it impacted the way I work – just that it set a stronger foundation than what I’d had before. And also, it wasn’t a “traditional” session, with someone talking in front of hundreds of people. There were a dozen or so of us in a fairly small room, and Bogdan was writing on a whiteboard.

Whereas, a session from TechEd Australia 2005 impacted me a lot.

It was my second TechEd event (the first was 1999), but the first after I learned the significance of the technical community. And I’d become a regular presenter myself, just not large events yet. I couldn’t tell you about many of the sessions I attended (I learned a lot more in the expo hall that year), but one session certainly had an impact. It was about security, and the speaker was Microsoft’s Jesper Johansson. LinkedIn tells me he now works for Walmart. https://www.linkedin.com/in/jesperjo

Of course there were things about security that I learned that day, although I couldn’t tell you much about that. The impact that he had on me was more than that. He held the audience’s attention like few technical presenters I’d seen before. I had done user group presentations and spoken in church plenty of times. I understood the need to connect with an audience, but Jesper validated the significance of that for technical sessions. I could remember university lecturers who connected with the room, but mostly that only applied if you were already interested in the subject matter. Jesper was attracting an audience who didn’t even care that much about security. He was leaving a mark. People were learning. Really learning.

Jesper’s session inspired me to aim for a style that would engage people. Technical sessions that would be interesting to people who didn’t necessarily have a background in the material. I wanted to be able to grab the attention of the people in the room (or at least most of them), and hold onto it. To teach their hearts and not just their minds, hopefully so they would leave changed. I try to do the same with my consulting, with the people I encounter in life in general. Of course I’m not always successful at that, but I do ask myself what people need from me whether I’m teaching a large room, a small room, or an individual. Sometimes it’s just a one-on-one conversation, but I’m there for the audience. I’m not trying to perform to be entertaining or interesting to them, I’m genuinely interested in the people who are there and I want to create that connection.

I know that my first TechEd Australia presentation didn’t achieve that. But I hope some of my presentations since then have. I try not to be bound by a slide deck or script (although I’ll have an idea about how the talk will go, and may have even used slides during the preparation), and instead reach into the room and be led by how the people are responding (definitely harder for online presentations, I know). I learn my material well, so that I have the freedom to explain it in ways that work that day, whether it’s through demos, whiteboards, flip-charts, or something else. It means I’m not a typical presenter, and while some audience members don’t like my presentation style, I hope that enough of them do. I’ve had a lot of good feedback from a lot of sessions, and even seen some other presenters adopt similar styles with great success.

I’m keen to see which sessions other people mention in response to Steve’s blog party this month, because I feel like there’s always something to learn about presenting. When I hear people saying that someone blew their mind, I want to know what it was that caused that. I would encourage every conference presenter to learn from the posts that respond to Steve’s question, and to inspire to blow people’s mind, to change their lives, and leave them different. I mean, why wouldn’t you?

@robfarley.com@bluesky (previously @rob_farley@twitter)

The post Presentations with impact! appeared first on LobsterPot Solutions.

I’m sure you know I’m a business owner. I’ve had my company since 2008. Sure, there have been ups and downs over that time, but in general it’s been good. We take on clients who want our services, but also try to maintain a level of flexibility with our time so that we can be responsive to whatever happens. Even if I’m spending most days at a particular client, I try to make sure I can respond to other things. Because just like how an employee might get sick, another client might have an urgent issue and I like to be able to respond. I like to be able to pause what I’m doing if necessary. Maybe this is to avoid risk, by not feeling like I have all my eggs in one basket. But maybe it increases the risk, because I’m less inclined to take on long-term full-time contracts.

Before I had the business, things were different. I craved stability, and I’m sure my career suffered for it.

I finished my university time in Melbourne at the very end of 1997. I’d had jobs as a research assistant in each of the three faculties I got degrees from, and it all wound up on New Year’s Eve at the end of 1997. I handed in my last pieces of work and after the long weekend I started working as a consultant, in a job that I had been offered six months earlier. I turned down the offer of a PhD for the stability of a wage.

In 2000, we moved to the UK with our two sons, and that felt risky. When we arrived I did not have a job to step into, but within a week I was deciding which of two I would take. Both were permanent positions – I was still risk-averse. I didn’t want to go contracting. I didn’t back myself enough. Financially, I could’ve made better choices, but I had a young family, and money has never been a goal. In 2002 we moved to Adelaide (where my wife grew up), and I still wanted permanent roles.

It took more than ten years of working in permanent positions (’98 to ’08) for me to take the career risk of starting my own company.

Someone called me an entrepreneur recently. I disagreed with them. I’ve never sought funding from anywhere. I’ve never tried to build up a company to sell it. I’ve always pushed against hiring a salesperson and never had anyone in the company with a sales-related title.

What I have done is lean into the community. I’ve done work for charities for little or no revenue – I even remember charging a client less than the hourly rate I was paying a subcontractor to do the work. And I’ve volunteered my time (and that of my employees) to work among the data community, running events, speaking at conferences, helping community members with what they need, even sometimes people who would consider us competition. And I’ve tried to listen to what God is saying. That’s been hard a lot of the time, but I try.

Is that a career risk? Sure – community work is definitely a risk. It would be better for me to focus on the bottom line and limit how much we do that isn’t tied directly to revenue. Community work is not how people are supposed to run a business. I’m in a group for business leaders and I know that I’m not the same as most of the others there.

This stretches to other parts of my life as well. In the comedy community, I’m hoping to start another open mic room again (I ran one in May/June last year before the venue closed). And I’m not running it to make money – I’m doing it to provide more options for local comedians. People need stage time to improve, and I can help with that. That might help me improve as a comedian too, but that’s not it. There is a need for more open mic space in Adelaide, and I think I can help. Is that a risk? Nah – it’s not taking anything away from my livelihood. But people have called me “commercially naïve” because of it.

Like I wrote further up – I’m not an entrepreneur. I’m good at helping other people solve their stuff, which makes me a good consultant. But running my own company still feels like a career risk to me. At the end of the day though, I like that my company reflects who I am, and I think the bigger risk would be for me to be someone I’m not.

@robfarley.com@bluesky (previously @rob_farley@twitter)

The post Career risks? appeared first on LobsterPot Solutions.

Now, as much as I’d like to wax lyrical about how English ages and changes over time, and how people nowadays would’ve said “don’t @ me”, I’m going to fit into Pat’s brief slightly better and talk about code ageing – T-SQL code even!

I could write about how new features in T-SQL can help, such as the NULL-aware “IS [NOT] DISTINCT FROM”. If you have code that uses the non-sargable ISNULL function on a column, then that might be something to consider when you next review that code. But I’m going to talk about T-SQL that can age badly because of more sneaky reasons.

There’s a truth that I think many coders don’t really appreciate, and that’s that a T-SQL query is rarely good by itself. Sure, there are queries that have strong potential to be good, but performance depends on a few other things too. And this can mean that a query that was good when it was written might have become bad over time, even though it’s the same query it always was.

Bad plans in the cache

One obvious example is when a bad plan has crept into the cache, and now every time that query runs, the bad plan gets chosen, and performance is now rubbish.

Plans show us how the Query Optimizer has decided to run a query. And I’m saying ‘decided’ to personify it a little. Of course it’s just maths, following a precise algorithm with heuristics to try to make the best choices, but at the of the day, as much as we know that there’s no arbitrary decision-making going on, it can really feel like there’s some tiny person in there, choosing to punish us for every mistake.

One mistake that gives us a bad plan can be simply calling the query with a parameter that is rarely used but causes the query to run a very different way. That plan could get cached, and suddenly every subsequent call for that query (with more typical parameters) uses a less-than-ideal plan.

Imagine the query:

SELECT TOP (100) t.Col1
FROM SomeTable t
WHERE t.Col2 IS NOT DISTINCT FROM @p1
ORDER BY t.Col3 DESC;

There are two obvious ways that this query could run. One would be to apply a TopN Sort on the results of a Seek on Col2, the other would be to perform an ordered (backward) Scan of an index on Col3 with a residual Predicate on Col2. Which one is better? No idea. It depends on how selective the filter is. It also depends on whether particular indexes are available, although right now I’m going to assume there are covering indexes that could produce both plans, one on “(Col3) include (Col1, Col2)”, and one on “(Col2, Col3) include (Col1)”. And it depends on whether the SQL engine knows about the selectivity of different values (ie, the statistics). All this is where the strength of a query is not just about how the query is written, but about indexes, statistics, the cardinality estimate model, and more.

If the table has 100M rows and 10% of them have the value we’re looking for, then the perhaps the best option is the ordered Scan. On average we might need to look through 1000 rows to find the top 100, if those values are mostly around the largest values of Col3.

But what if only 0.001% of the rows have the value we’re looking for. A total of 1000 rows in the whole table. Now we’d be better off using a Seek to find the 1000 rows. But that Sort might need a large memory grant.

And what if there are 10M of them, but they’re all around the smallest values of Col3… oh, the joys of query tuning.

Maybe the value is usually selective, but every so often it’s not – maybe when it runs at 4am it uses a value which isn’t selective, so I’m willing to take the hit on performance, but during the day I want it to use the Seek + Sort plan. And usually this is fine, and there’s a cached plan that does the Seek + Sort that gets reused every time. Terrific.

Then one night, there’s an index rebuild, and the next time the query runs it’s the overnight non-selective parameter. The compiler comes up with a plan which is good for that value, and it’s the ordered Scan plan, which gets cached. Then when users arrive in the morning, that’s the plan that gets used every time, and it’s slow. Agonisingly, it gets often (but not always!) fixed by restarting the SQL service, and let’s stop imagining how much pain organisations can get into when they start down these rabbit holes.

Could we force the Seek? Sure. Should we force the Seek? That’s a tougher question. Maybe we need the overnight query to run better too. Maybe we need two queries. If the parameter value NULL is the only one which is non-selective, then making two queries is easy. We could test the value for NULL and then run a different query. That’d be easy to do. Or maybe it could be fixed by using the OPTION (RECOMPILE) hint, but that’s not exactly free either.

Inside knowledge is great, but perhaps we didn’t have that knowledge when we wrote the query, and the code has aged badly because we didn’t anticipate the future.

There are a bunch of other changes that can cause a perfectly adequate query to age poorly. Some quick examples:

More data than expected

It doesn’t have to be a ‘bad plan’ scenario for a query to be less good than when it was written, because conditions can simply change. The system was so popular than the amount of data grew and now some process takes a lot longer than before. The scan was fine, but now the table is a lot larger, and the values that we thought would be selective aren’t (or vice versa).

Or maybe because the volume of data has increased, the number of physical reads is higher. Perhaps adding RAM is a simple solution, but maybe queries that were fast enough with full scans need to be rewritten.

Better (!) estimates

Maybe the version has been upgraded, and the cardinality estimates are improved. The Optimizer used to think that 20 rows would be the results of a filter, but it was always wildly wrong, with 20,000 rows being more typical. Now with a later version of SQL, the estimate is correct, but because of this, there’s a join that’s using a Hash Match instead of a Nested Loop. Way fewer logical reads, but now it needs a larger memory grant to run the query, and system performance is hurting.

Concurrency

Speaking of memory, maybe the concurrency requirements have increased. Maybe there are 100x as many as expected users trying to use the system at once, and those memory grants are causing a bottleneck. Maybe queries need to be tuned to discourage large memory grants.

Index changes

Maybe partitions have been introduced, and the indexes haven’t been reviewed.

Or maybe a well-meaning DBA or external consultant (ahem!) has been brought in and a new index has been added to reduce the reads. Except that now some other query has started to use that index and although the Query Optimizer has good reason for choosing that index, it’s slower than it was before, or that query we’d tuned to reduce the memory has returned to its old patterns.

So… what should we do?

Naturally what we need is a crystal ball, and write our queries (and our indexes, partitions, statistics, hints, whatever) according to how we expect when things have changed. Typically if you write with scale in mind, you won’t be impacted too much by scale.

But also, understand that there will be some things that you can’t expect so easily. You can’t necessarily expect to anticipate that a particular index will be created, and I’m definitely not suggesting you force the use of a particular index from day 1 of your query, because the change might be about the shape of the data – something that was marvellously selective no longer is for example.

The general feel of T-SQL code might not age as obviously as, say, .Net application code, with techniques that have been superseded over the years. Or a UI that that has aged like a trendy haircut. A lot has changed in the database world, but a T-SQL query now looks roughly the same as it did in the days of SQL Server 2005. But you should totally expect that a query you wrote in a small dev environment might behave quite different in production as the size and shape of the data increases.

So don’t beat yourself up about it – just schedule a code review, or at least a good tuning session. There’s bound to be some things you can fix up, and maybe those fixes won’t age nearly so badly, because you have a better idea about how things will look in the future.

@robfarley.com@bluesky (previously @rob_farley@twitter)

The post Ageing code appeared first on LobsterPot Solutions.

The name of this post “Really, Louis?” is nothing to do with how Louis thought for a number of years that Arnie Rowland was me. But rather about how people (especially “conscientious types” (that’s only ‘code’ if you want it to be) like so many of us in the data world) frequently take ourselves back through our mistakes, beating ourselves up about it, thinking about how we could’ve avoided it, what we could’ve done better (before, during and after), as if time machines could exist.

I mean, sure, there’s a school of thought that because we learn through our mistakes, we become stronger. But you know, I’d rather learn through near misses. “Oh, that could’ve been bad, let me code up something so that doesn’t happen” is a better experience than having to learn when a disaster has actually happened. And hands up if you know that having a good ‘disaster recovery’ plan only saves you from certain problems, not all problems… thank you, I see that hand. And that one. Yup. All the hands.

There’s an article on Psychology Today called “Coming Out of the Shadows of Our Worst Mistakes“, by Rodney Luster, PhD (he has a blog called “More Than A Feeling” which has a lot of great posts and is worth looking at if you have mental health struggles). One of his points is that even a single mistake can heighten anxiety and depression symptoms. And still, we all make mistakes. All of us. Data professionals included.

Our mistakes shouldn’t define us. We can quote Sting when he sings that “history will teach us nothing”, but we hope that each mistake makes us likely to fall into the same patterns that led us into trouble before. I heard something recently that says our histories might describe us, but they don’t define us. And I want to assure you, reader (and myself), that your history is more about successes than your failures. You wouldn’t be where you are today if you weren’t good at what you’d do. (You would’ve been promoted into management instead.)

As Dr Luster writes, our mistakes can cast shadows over us. Over our mental wellbeing. Over our general mood. We can feel guilt and shame about mistakes, and it can take proper effort to feel redeemed. As a Christian, I know that this is a major aspect of Christianity. The Bible says that we’ve been set free from our guilt and shame, and God loves (and likes) us despite our failings. And yet, we relive our mistakes. Especially the ones involving those areas in which we know we’re meant to do better. Where we’re supposedly ‘experts’.

I’m not going to dwell on my mistakes today. Well, I’m not going to dwell on my mistakes any more today. I’ve already spent time doing that, even though I knew it wasn’t a good idea. I thought at some point in my life I would work out how to stop reliving mistakes, but that’s not a skill I’ve learned yet.

I say to my clients, “Don’t worry – we’ve all done it”, and “You’re not a proper data professional until you’ve stuffed something up”. For some reason, I still remember a question in the Microsoft Certified Master that dealt with corruption, and (without going into any detail, because NDA, and because I really don’t remember the detail) failing to grab a copy of the database files first. That’s not a mistake I’d ever made with a corrupt database in the real world (and there had been plenty I’d handled – I think I had a reputation for fixing corruption, back when it was more common than nowadays), but in the exam – yeah, made the mistake. Maybe it was because it was an exam. Maybe it was because I was sitting by myself (not alongside a customer). That day in the exam, I found myself wanting those files (because of another mistake!), and didn’t have them.

The consequence of my mistake in that exam was basically nothing. I passed the exam and earned my MCM status. Apparently I scraped through with the lowest passing score I could get. But I still passed. That corrupt database question didn’t matter at all. And yet, every so often, it appears in my thoughts. I wonder whether I would make that mistake again, perhaps in an environment where it would matter more. So you can imagine how much I relive the mistakes that actually count for something…

The mistake I’m going to relive for you/Louis is the mistake of reliving my mistakes, as I’ve been describing. I know it sounds meta – but I learned recently that ‘meta’ is the Portuguese word for ‘goal’ (not the football goal, but the thing you’re trying to achieve), so maybe it’s okay that it’s meta. It’s a useful goal to stop reliving your mistakes, even if it is meta.

My advice to you is to learn forgiveness. Forgive everyone else their mistakes. It’s easy to forgive people their mistakes because, by definition, they’re unintentional. So do that, and get used to doing that. Then work out how to forgive yourself for your own.

…and teach me how to do it… 😉

@robfarley.com@bluesky (previously @rob_farley@twitter)

The post Mistakes? Really, Louis? ;) appeared first on LobsterPot Solutions.

I’m in a business leaders group called Vistage. In Australia it used be to called TEC (I’m quite happy about the rebranding because “TEC” sounds like “tech”. Plus each group has a chair, and “TEC Chair” sounds a lot like “deckchair” to me). Anyway, when there’s a discussion going on, my Vistage chair sometimes says, “If you knew the answer, what would it be?”, which comes from Sir John Whitmore’s book “Coaching for Performance”.

It’s an annoyingly good question because it shifts the perspective a little. It’s not just asking what the answer is (to which “that’s the problem – I don’t know!” is a common response), it’s asking the person to imagine a situation where they already know. It subtly creates courage in the responder, because they’re asked to imagine that they know and that what they know is correct. I say it’s “annoyingly good” because it’s essentially using mind games to get us to move us past not knowing.

Mike’s asking us to write a note to ourselves from ten years in the future is framed similarly. He wants us to imagine a world in which we’ve got through the next ten years, and we now have the benefit of hindsight. He wants us to look back at 2015 with hindsight, to point out that a thing we were going through at the time would work out okay (or maybe the opposite), and then give ourselves similar advice about the thing we’re in now. Essentially, he’s asking, “If you knew you were going to be successful, what would you do/feel differently?” He wants us to prod ourselves into action about something in the world.

For me, I’ve just turned 51. I remember turning 31 and finding it’s still one of the more jarring age-changes I’ve had. Something about changing from “thirty” to “thirty-something” felt like I had entered a different phase of life, and I’m trying not to feel similar this year now that I’m “fifty-something”. Looking back at being 41, and imagining what it might be like to be 61, it’s interesting.

Ten years ago the world felt like it was changing quickly. And now it feels like it’s changing even quicker. In another ten years, I suspect the rate of change will not have slowed at all, although there is a significant chance that a breaking point might have been reached along the way. Ten years ago we had no idea of the events that were coming. Australia had seen five changes of Prime Minister in the previous ten years, but despite only having two more changes since then, the impact of fires, lockdowns, and other global events have made the world seem vastly different.

My advice to 2015 me is to ride with the big waves and to not sweat the small ones. I don’t think I did let the small waves knock me around, but I know there were times that were harder than I’d like. I’m still here though! Data is still data, whether the tools change, or the trend is towards or away from relational, or whether we’re working on-site or remote. Basic needs of society are still the same, and businesses still need their data to be good, insightful, and well-looked after. New ways of doing things might change the day-to-day activity, and people might be dealing with jarring stresses from the world more than ever, but the core needs of society and data-focused businesses are mostly unchanged.

I suspect I’ll feel similar in 2035 looking back at now. I have no idea what will happen in those ten years. AI is bringing change, bringing new tools (as I wrote in August), but I don’t think it will change what people or businesses need – just how we deliver them. People still need people to be there for them, and businesses still need to trust their data and have insight into it. This year I passed 20 years of running the user group in Adelaide, and in another 10 years I might have just gone past 30 – which will be just about half my life by then. The pandemic caused a lot of change to community, and even within the local user group things feel different now to then. But people do still need community, and I will try to help make that happen, whether that’s in the technical community, the comedy community, or the church community.

So the advice I’m going to give myself from the future is to make the right decisions. Keep doing the things that I should be doing, and find the right balance between pushing myself and going easy on myself. Tonight is a case in point – my back is sore, but it often is so I shouldn’t let that stop me going out to Monday night open-mic comedy. I haven’t been in a long while, but I need to get that back into my weekly schedule.

If current me knows things are going to work out, what will I have been doing to help that? Yeah – doing those things that I know are what I should be doing. I’m pretty sure hindsight will thank me for making those good decisions.

@robfarley.com@bluesky (previously @rob_farley@twitter)

The post Notes to self… appeared first on LobsterPot Solutions.