Dr Lau is an associate consultant in the O&G Department of Kwong Wah Hospital.
When we started working on roster scheduling in September 2024, he was spending at least 4-5 hours per month scheduling rosters for his department. They had a complex rostering structure comprising from interns, 1st, 2nd and 3rd calls.
Not only that, the rules are complicated with many aspects playing important roles simultaneously. This is further complicated by multiple staff requests, annual leaves and much more.
Here’s how we used mathematical optimization to help with roster automation for his ward.
The Challenge
The O&G Department roster consists of 5 levels of seniority, from interns, 1st, 2nd, 3rd, to consultant calls. Dr. Lau is responsible for the scheduling of 1st, 2nd, and 3rd calls, consisting of around 27 people.
Here are the rules and objectives:
Rules
- 1 Staff is needed per day for every seniority.
- No one can work more than 1 call every 3 days.
- Everyone get’s a post-call day-off.
- No calls on Annual Leave.
- Everyone can only get at most 1 call on Saturdays per month.
- Because Saturday calls are the most worst calls.
- Staff must work a fixed number of shifts on each seniority, on weekdays and weekends.
- This is in line with the department’s requirement.
Objectives
- Satisfy no call request and on call request, aiming for 100% satisfied.
- Try to pair up junior 3rd call with senior 1st or 2nd call (with 2nd call more preferred).
- Because teamwork is important in O&G emergencies, and a junior 3rd call could really benefit from a more senior subordinate.
- Disperse calls to once per 5 days as much as possible.
- To balance out workload across the month.
- Evenly distribute thursday calls.
- Because it’s the most wanted across all members — you get a long weekend.
- Junior 1st calls should have a dispersed pattern of seniors.
- Because no one wants to get called with the most junior 1st calls, they should be evenly dispersed between seniors.
As with our other clients, Dr Lau has also experienced times where scheduling is frustrating.
“The further you proceed with a schedule, the fewer possibilities there are. There have been one time where I discovered it wouldn’t work at the very end, and had to scrap all the work and re-make a schedule from scratch.”, said Dr Lau.
We aren’t the first solution Dr Lau tried, either. He previously tried roster automation with a HA rostering program, but it didn’t work as it didn’t take into concern of all the above. This is why he reverted back to manual.
And this is also why he didn’t have high hopes for our algorithm to work for him.
Well, it’s time to change his beliefs.
Roster Automation is Ideally Suited to Set Up Specific Rules
Unlike no-code solutions in the HA, you can specify requirements to the tiniest detail with roster algorithms, because they’re basically mathematical models that can be molded according to whatever the requirements are.
In contrast, HA’s roster automation often assume lots of things: they assume an 8-hour shift, they assume a seniority level that cannot be sub-divided, they assume you don’t need the “human elements” of scheduling…
This renders the SRS unable to tackle the fine-grain requirements that are needed in real-life scheduling.
But it does make sense — SRS was originally developed to monitor work hours of doctors and generate work reports for regular audits, not to craft the perfect schedule that works for wards.
Because of that, we will employ mathematical optimization to help Dr Lau with roster automation.
Categorizing the Rules and Objectives to Map Out the Mathematical Model
Broadly, the requirements of a ward roster can be classified into 2 categories:
- Constraints: Rules on the Mathematical System that cannot be broken.
- Objectives: Mathematical Terms to Optimize for during the scheduling process.
While objectives can be further classified into 2 subcategories:
- Goals: Metrics to Maximize or Minimize during the scheduling process.
- Soft Constraints: Conditions to prevent breaking during the scheduling process.
Now, the naming of these terms are indeed quite confusing at first glance: especially the fact that “soft constraints” falls under “objectives” rather than “constraints”.
But let’s break it down with some examples.
Examples of Constraints
These are the bread and butter rules that make up the roster. They’re never expected to be broken, and if they’re broken, it’s probably something to do with the model set up.
E.g. 1 staff is required for each seniority every day.
If you break this rule, you basically either overstaff or understaff your ward. This is NOT supposed to happen.
E.g. No call can be allocated on AL.
How can a staff be both on-call and on annual leave? The AL should not have been approved off in the first place.
Examples of Objectives: Goals
These are the metrics to optimize for during the rostering process. You want to keep them either maximized or minimized, but do understand that you’re doing this in the constraint of real-life rules.
E.g. Evenly distribute all Thurday Calls.
Mathematically, we’re actually minimizing the “maximum absolute deviation” of the number of thursday calls allocated to each person.
At best, we want the deviation to be 0 — so that it is so truly evenly distributed. But owing to the number of staffs, some people taking AL, etc, you can only do your best.
E.g. Spread out calls as much as possible.
This correlates to minimizing a term that gets smaller and smaller as the person’s call gets more and more dispersed across the month.
Again, you don’t expect it to be fulfilled for everyone. Ever once in a while, you might be forced to have a person take 1 call every 3 days, but even if you do it for once, you don’t want them to do it continuously.
Examples of Objectives: Soft Constraints
These are constraints that you’d like to not break whenever possible, but if you do have to break them, you try and keep the number of broken constraints as low as possible.
E.g. Junior 3rd Call should not take the same call as junior 1st call and junior 2nd call.
Because junior 3rd calls are new to this position, they might not be able to handle O&G emergencies on their own — they need a more senior team around.
This is something that is easy to quantify — if this rule is broken, the solver should be penalized by a set weight, so that it tries not to break the rule as much as possible.
So, the difference between a hard constraint and a soft constraint is that, if a hard constraint has to be broken, the solver will return a “infeasible” status and force you to change the ground rules, but if a soft constraint is broken, the solver will record that, but will still try to find a best solution with that constraint broken.
Deciding between Soft Constraints and Objectives
There are 1000 ways we can model for Dr Lau’s ward, however, the most debatable part of the modelling is probably deciding between soft constraints and objectives. Let me explain.
We want to pair up junior 3rd calls and senior 2nd, senior 1st calls
How do we model that? Mathematically, there are 2 ways:
- Model as a Goal: use linearization techniques to couple junior 3rd with senior 2nd/1st calls, then try to maximize the number of couplings.
- Model as a Soft Cosntraint: set up soft constraints to prevent junior 3rd from taking the same call with junior 2nd/1st calls.
How do we decide which is better?
Well, both works fine, but we ultimately chose to model the constraints as “soft constraints”. Here’s why.
It is easier to monitor breaking of soft constraints
For a generated roster, it is much harder to monitor the satisfiability of goals, compared to the breaking of soft constraints.
For example, if we are to pair up the couplings between junior 3rd calls and senior 2nd/1st calls, we’d have many terms to monitor for each night (# junior 3rd * [# senior 2nd + # senior 1st]). This is in contrast to the breaking of soft constraints, which we can go back directly to the list of soft constraints, and check against the violation of each constraint.
In addition, in general, the solver does a really good job in pairing up these constraints. Therefore, it is actually easier for us to report on the violations compared with reporting on the satisfied goals.
We also want to maximize the shift requests
There are 2 kinds of requests: on-call requests and no-call requests. Dr Lau wanted to achieve a 100% satisfaction of these requests — as per his usual practice.
In fact, it is balancing between these requests and making a quality roster that poses the greatest challenge, “It’s easy to just scramble a workable roster that doesn’t take everything into account, but that would be at the cost of my colleague’s work-life balance.”, says Dr Lau.
At first, we tried to model the requests as a linear sum of the list of requests matched. It produced a result that had a 10% on-call request satisfaction, and around 70% of no-call request satisfaction. However, we soon realized this was too low. Dr Lau’s ward usually practiced 100% satisfaction, so we had to tune down the weights of other objectives.
Eventually, we migrated the request matching into soft constraints as well, for better reporting and greater control, and we set the weighting of these constraints 50 times higher than other constraints. This eventually resulted into a 100% satisfaction of all staff requests (with the exception of 1 infeasible request due to violation of hard rules).
Results: Workable Roster Automation Within 2 Minutes
This engagement produced some of the most impressive results we’ve seen.
The total number of constraints needed to formulate Dr Lau’s ward was a whopping 36,000 constraints. The roster automation solver terminated with a result that is within 5% of optimal target, generating a roster that looked like the following (with names replaced by dummy).
We achieved
- 100% viable request math rate.
- 100% pairing of junior 3rd calls with either senior 2nd calls and senior 1st calls.
- Fair distribution of preferable Thursday Calls
- At most 1 Saturday Call per person
We also helped them meet all these targets within 2 minutes, in contrast to the 4-5 hours manual scheduling time. That represents a 150 times speed up, and a 99.3% time saved, with no risk of human errors, and automatic reporting of soft constraint violation.
It was a pleasure to work with Dr Lau and help them drastically cut down roster scheduling time. And, we were pleased to see that one of the key features Dr Lau highlighted is the need for a high degree of customization, which can indeed be done through mathematical optimization.
How to Work With Us or Learn More
- Our Service: If you want to hire us to build a roster automation algorithm for your ward, complete with a deep analysis for your ward rules, soft constraint violation reports, and optimized mathematical modelling, learn more about working with us here.
- Our E-Book: Individuals looking to learn our strategy of roster automation can download our e-book for free. It is a hand-on guide for any programming beginner to start building their first, simple mathematical model for roster scheduling. You can then add in more complexity as required by your ward rules.
- Special Thing Brewing: As inspired by some of our clients, we’re currently building a fully customizable no-code solution that helps you build your own algorithm from a web application. We’re offering free trials in return for feedback and opinions. Contact us to learn more.