jonathan@researchcomputingteams.org
Hi, all!
This is the last part of the stop doing things challenge - the change management of actually stopping doing something. Or you can skip straight to the roundup.
The hardest thing about stopping doing things, of course, isn’t the identifying what things to stop doing, but actually stopping and staying stopped doing them.
There’s really only three major steps to managing any substantial change, whether to something you’re doing or your team is doing - communicating the change, implementing the change, and monitoring the change. For stopping doing something, implementing the change is mostly trivial - just don’t do the things! The other two steps are things that you have to continue doing, until well past the time you feel confident that the change has “taken”.
So for stopping something, the steps will look like:
The first phase is to speak individually with each of the people who will be affected by the change - your boss, stakeholders who were making use of what you were doing, and if applicable, team members who were part of doing that thing. The framing is that you have a plan for the coming year, and you’re getting input on the plan. Walk them through a pretty thorough proposal of the change - why you’re doing it, how it will happen, and alternatives for them.
Throughout this conversation, genuinely seek input and feedback. These conversations are a pre-emptive way of saying no to stakeholders, an opportunity to align you and them to the same sets of goals. The input doesn’t necessarily have to be positive, and that’s ok. You’ll find stakeholder problems you hadn’t anticipated, which you can then make plans to mitigate; you’ll find things you thought would be problems that in fact aren’t, so you can make your plan a little simpler. You might also find some people who are just dead-set against what you’re proposing, which you’ll likely have to discuss with your boss.
Having these individual conversations serves three extremely important purposes:
You can greatly improve the effectiveness of these conversations by sending the person you spoke with an email fairly promptly after each conversation, with your notes on what you heard from them in your own words, what steps you’re planning to take based on their input, and asking them to let you know if you’ve missed or misstated anything, or if they have anything to add. This significantly strengthens all of the intent, trust-building and feedback-collection purposes of the conversation.
Depending on how serious some of the objections are, you might have to make nontrivial changes to your plan, in which case a second round of individual communications might be useful to keep them up to date. That also has the huge advantage of showing that you take the feedback you are gathering seriously.
(I’m embarrassed to say that I used to completely skip this step and leapt straight into detailed planning and public announcement of the proposal, which is a completely unnecessarily and unnervingly efficient trust-shredding exercise, to say nothing of being, frankly, arrogant. Pre-wiring the group communication, is an old idea that I first heard from Manager-Tools, and although it sounds a bit obvious in retrospect, it was a revelation.)
Once you’ve gathered the input and updated your plan, the next step is to choose a simple way to monitor that you are in fact stopping doing the thing. The biggest thing you have to worry about is backsliding. People who used to do certain tasks will tend to revert back to what they know, even if they know intellectually your team isn’t doing that any more. “Just this last time”, “I wasn’t doing anything this afternoon anyway”, etc. That goes for you as well as your team members!
A simple way to keep on track is to have some kind of monitoring system, which both communicates that the change is something that matters and sets up a process whereby backslides are visible and so people (maybe you!) are accountable. Metrics don’t have to be hard or technically complicated; you can start by just, even manually, counting the things you care about. You can count the number of data science projects the team is currently working on, and the number of “off-topic” projects, and publish them - watching the numbers track up and down, respectively. You can show the number of jobs on the system to be decommissioned, and watch it trend downwards as decommissioning nears and alternatives are found.
The data can be manually collected, and they can be communicated manually - at your weekly team meetings say; the point isn’t to have a flashy dashboard, the point is to have a simple, visible indicator which is going the right or wrong way.
Once those things are lined up, public announcements and tracking can start. You’ll have to keep communicating, and you’ll have to keep tracking, but with the foundations of individual communication and simple indicators in place, the likelihood of the change taking root and persisting are much greater.
And that’s it! Now off to the roundup.
Your Star Employee Just Quit. Will Others Follow - Art Markman, HBR
Maintaining a strong team isn’t an activity that ever stops. We need to actively, constantly, be building the team - by supporting team members development and career goals, by giving them new challenges, and by bringing in new team members or developing and keeping an eye on a “bench” of possible candidates.
It’s not necessarily indicative of a problem by itself that the member is leaving - it’s good and healthy for people to move on into other roles. We should be actively helping people prepare to take on more responsibilities, including new roles!
But when a key team member leaves, it can have cascading consequences. Workload goes up, a trusted peer is gone, and remaining team members now feel uncertain about a workplace that previously felt pretty solid.
Markman recommends doing a thoughtful exit interview with the outgoing team member. It’s a valuable opportunity to get perspective of what you and the team can do better, from someone who can afford an extra bit of candour as they leave. It’s also important to assess if they were “shields down” to other opportunities for reasons that we could have influenced.
Then he recommends doubling down on things we should be doing at some level all the time anyway:
Key people leaving is one of many bad things that it’s best to have some kind of contingency plans for. I’m thinking of building a list of scenarios like this and emailing/tweeting them out weekly as a desktop planning exercise - helping research computing managers ensure they have plans and playbooks laid out for various contingencies. Catastrophe-as-a-Service sort of thing. Is that something you’d be intersted in?
How to Manage Multiple Projects at the Same Time - Elizabeth Harrin, Girl’s Guide to PM
Many of us in research computing are managing multiple projects/efforts simultaneously; Harrin’s post reminds us we’re not alone, and that most (59%) of project managers lead 2-5 projects. Typically our projects are part of a program - a portfolio of projects that are complementary to each other - which makes some of the people things easier, in that the teams overlap and see common goals.
The four big areas Harrin sees to keep an eye on are:
Harrin doesn’t offer any silver bullets for these areas, because there aren’t any, and what works best for you will depend on your preferences and situation.
For managing your own tasks, I keep a weekly priority list by my desk of things to keep an eye on, with categories for “issues” (efforts I have to actively manage) and “monitor” (efforts that are going well and I want to make sure they don’t become issues) to keep myself honest and not neglect the tending to the “monitor” areas.
Similarly, moving people’s efforts across projects is tricky, for us and for them, and again there’s no real silver bullet here, but it helps that our work usually complements each other across different efforts.
Job Interviews Don’t Work - Farnam Street Blog
Traditional interviews are pretty crummy for identifying good candidates for jobs - we miss too many potentially good matches and let too many potentially poor matches too far along the pipeline, wasting their time and ours. The blog post points out in some detail some of why that is:
We actually know a lot about how to make interviews better; the article suggests:
5 Tips for Saying No To Stakeholders - Roman Pichler
User power, not power users: htop and its design philosophy - Hisham H. Muhammad
You can’t keep focus on your goals and priorities without saying no to requests. We’ve covered articles on this in the roundup before, and alluded to this in the stopping things advice at the beggining of the newsletter, but it’s an important topic! Pichler emphasizes:
All of these things are important in both individual and group communication of changes, whether preemptive (“We won’t be doing this any more”) or reactive (“No, I’m afraid we can’t do that.”)
A nice example of this in a software development context is Muhammad’s article on saying no to a feature request for htop (which is a lovely package). htop had a very specific vision, and user requests contrary to that helps clarify the vision if it’s taken as an opportunity to say no, rather than muddying and compromising the vision through indiscriminate yeses. The story in the request is a pretty good illustration of all of five of Pichler’s points.
Illuminated equations - Matt Hall
We often have to communicate pretty complicated material to an audience who may be extremely capable and knowledgable but not necessarily in exactly what we’re talking about. Here Hall gives several examples of explanations of equations - from heavily annotated equations, to dynamic explanations, to multi-media explanations.
Minigraph as a multi-assembly SV caller - Heng Li
One of the reasons why technological readiness, or maturity, is a much better way to think about most research software development than technical debt or even sustainability is that most research software development is in the early, discovery, stages of readiness:
Most research software development projects never really gets past early stages, because in research software development, like any other aspect of research, most attempts aren’t as fruitful as hoped. And there’s nothing wrong with that; that’s just how discovery works. This article by Heng Li about the evolution in use cases for minigraph, a bioinformatics tool he wrote for mapping sequencing reads to graph genomes, is a good case in point. It was pitched as a fast caller for structural variant calling, but:
My exploration took a turn when one anonymous reviewer asked me to check the LPA gene. It was not in the graph because the gene was collapsed or missed in all input assemblies. Fortunately, I had several phased hifiasm assemblies at hand. LPA is there and minigraph generates a complex subgraph (figure below) far beyond the capability of VCF. Then I realized what minigraph is truly good for: complex SVs.
Now, I’m not sure the author would agree with me on this next point - he’s a very careful software developer who only releases solid, performant, code. But in general it’s difficult to make sound decisions about optimization, what is and isn’t technical debt, or what’s necessary to turn the prototype into a sustainable, maintainable body of code, when even the code’s use cases aren’t clear yet. And for all software, but especially research software, users will find use cases you haven’t dreamed of.
Between each of those rungs on the ladder of research software technology readiness are more user input leading to more potential pivots and more data about what is needed and how to weigh tradeoffs. Sustainability and technical debt are only considerations at the very top rungs of the ladder.
New codebase, who dis? (How to Join a Team and Learn a Codebase) - Samuel Taylor
Whether we as managers or team leads move to a new team, or are welcoming someone new to your project, it’s good to have a plan for how to get familiar with the new code. First is understanding the big picture:
And then once that high-level picture is understood, get into the code and start doing something:
I continue to think that for those of us who came up through the research side of research computing, our research experience gives us something of an unfair advantage of being thrown into situations where we have to develop an understanding of a new body of knowledge. Absorbing lots of new information, forming hypotheses, and testing them plays pretty strongly to our training.
Doubling down on open, Part II - Shay Banon, Elastic
A lot of infrastructure code like databases have a business model where the core product is FOSS, and they make money by selling licences to a premium version and hosted services. This has worked pretty well up until recently, but now companies like AWS and others are also running and hosting these services for a fee, while making changes that aren’t contributed back. The reason is that running the software as a service isn’t “distributing” the software, so changes can be made and kept proprietary while still profiting from the software development and cutting into the money it takes to provide the original software development.
As a result, late last year it was announced that Elasticsearch and Kibana are moving away from Apache 2.0 to dual licensing under their own Elastic License and MongoDB’s Server Side Public License (SSPL). This gives users most of the same rights, but adding the restriction that if you run it as a service you also must release code changes back to the community.
(Elastic is getting a lot of heat for this, with claims that it’s “not an open source licence”. A lot of that is unjustified - this isn’t Common Clause, which had bigger issues.)
For some - not all, but some - research software it may make sense to have running the hosted software, fee-for-service; researchers could pay to have the software run rather than pay (directly or indirectly) for the hardware and run it themselves. For such tools it may make sense to consider these modified licenses to ensure that contributions go back to the research community that could benefit from them.
Profiling GPU code with NSIGHT systems - Jorge L. Galvez Vallejo
NVidia is in the process of replacing the previous nvprof tools with a unified set of Nsight tools - Nsight systems, Nsight compute, and Nsight graphics. This article by Galvez Vallejo shows how to use Nsight systems (nsys) to show the high-level overview, and Nsight compute (ncu) to dig into the performance of a particular kernel.
Our Dumb Security Questionnaire - Jacob Kaplan-Moss
A “10 questions, no diagrams” questionnaire, intended for small early stage companies looking at vendors, but really a pretty good short set of questions to consider about any security practice.
Announcing the Security Chaos Engineering Report - Aaron Rinehart
Introducing the new area of Security Chaos Engineering - introducing faults and failure modes to test your system’s or application’s security. If your security depends on having no failures, then you have a security problem.
Shifting Cloud Security Left — Scanning Infrastructure as Code for Security Issues - Christophe Tafani-Dereeper
Tafani-Dereeper evaluates several tools for automated security scanning of Terraform deployments. The tools are set up to look for typical misconfigurations for AWS, GCP, and Azure, but most support custom checks that could be used for, e.g., local OpenStack deployments.
The article evaluates checkov, regula, terraform-compliance, terrascan, and tfsec, on metrics of project activity, support for custom checks, and usability. Also shown is setting up a static analysis workflow for terraform deployments. Tafani-Dereeper recommends checkov or tfsec for getting something working fast, while correlating multiple resources is bet supported by terrascan or regula, while terraform-compliance has a nice behaviour-driven development language the the author particular likes for adding complex custom checks.
Searching for RH Counterexamples — Deploying with Docker - Jeremy Kun
This is the fifth of an interesting series of tutorial articles, using the search for counterexamples of the Riemann Hypothesis as a fun but likely not very fruitful use case, touching on test-driven python development, using a database, adjusting search strategies, and unbounded integers in databases. This article goes through the steps of dockerizing the database and search program, and deploying it, which is a good starting point if you want to get started with such things.
CrossHair is a python test suite based on contracts - like Ada! This is closer to property-based testing than usual unit testing, but not quite the same. It also has really interesting functionality like diffbehaviour.
For those who, like me, think IDEs more or less peaked with Turbo Pascal/Turbo C++, Turbo Vision is a text-based user interface library with a very different API than e.g. ncurses, and works on Windows as well as POSIX-y operating systems.
Interesting to see the growing HPC-as-a-Service options out there as HPE signs up with a Swedish facility for hosted HPC (a new Greenlake region?)
An interesting look at how TACC works and how the pandemic affected their 170-staff-strong centre.
A nice walkthrough on setting up a Jekyll website from scratch with simple.css and using Netlify to host.
Airtable is a nice easy databse-y backend for side projects; here’s a developer’s guide to developing using the airtable API.
A more functional line editor for bash - ble.sh
Powershell on Linux. I know this is heresy, but: future linux shells could learn meaningful things from powershell.
A data-aware shell that turns complex piped command lines into DAG workflows, and then migrates individual processes to be closer to the data(!!).
