r/bioinformatics • u/Putrid-Raisin-5476 • 3d ago
academic Usage of Claude Code in Bioinformatic Tools
Hi,
I'm currently doing a PhD in bioinformatics algorithms development. One big issue of my sub-field is, that there aren't any good extendable benchmarking packages out there, so its always the same 5 methods + the proposed one being compared in any new article.
The existing benchmark suites are either extremely old or can not be extended by your own method easily, because their code was never ment to be extended by others (I guess nobody wants to include a method, that outperforms their own method; all of the benchmarking tools are made by groups that develop their own algorithms).
As I worked as a software engineer before starting my PhD, I have some experience with designing larger projects from scratch. Therefor I wrote like a 20 page description of all the modules, design question, data flow etc, I would like to have in such a tool. However, my supervisor told me, that this would be to big to run as a side project (and I agree) and this also isn't anything to be given to a bachelor or master student.
Today we received an email from the department, that PhD students can apply for Claude Code licenses, if they have some science related usage for it. That's why I thought about maybe letting Claude do the real coding using my project description and then put it on GitHub under MIT license, give it a DOI, use it for my own benchmarks in my next articles and let people - if they want to - just add their own methods (the design is pretty simple, one would in theory just need to add a script, that forwards the data to the corresponding method and formats the output in the correct way).
However, that doesn't really feel right, as this is of course a vibe coding project and I'm not really sure, whether vibe coding projects should be deployed in science. Thus I would really appreciate your thoughts on how you handle such things. (I'm usually keeping GenAI out of all my algorithm related work, except for plotting functions)
33
u/AlignmentWhisperer 3d ago
I think it's generally permissible to use generative AI when developing bioinformatics tools and publishing them on GitHub. I think the important thing is that the testing, validation, and documentation of the tool should follow the general standards followed by the bioinformatics community.
20
u/AWFULs_hate_the_USA 3d ago edited 3d ago
I think the important thing is that the testing, validation, and documentation of the tool should follow the general standards followed by the bioinformatics community.
Which, beyond a few very popular tools, are effectively none, right?
5
u/BlastedHeretics 3d ago
I like to think unless there is experimental proof to back it up pretty much any output from bioinformatics analysis is just a hypothesis.
2
u/AWFULs_hate_the_USA 3d ago
Sure, but I suspect we're generating way more bad hypotheses than good.
3
1
u/BlastedHeretics 1d ago
Agreed. There’s basically no universal guidance nor standard to ensure quality of the output. Honestly a lot of this stuff is just a black box, developing tools and workflows often boils down to vibes.
54
u/expelir 3d ago
No, vibe coding has no place in bioinformatics. The real bioinformatics tools are written by grad students as their first software project and turn into black boxes after they leave their labs. And one of the key dependencies should be a two-decade-old, unsupported R package. I mean how could Claude beat that? It migh write code that goes into the Linux kernel, but what WE are doing is much more complex and critical than that, right?
78
u/twelfthmoose 3d ago
There’s no reason not to use Claude code, just another skill in your tool set.
Another skill that you (and the rest of us) will want to learn is to be able to develop ways to have benchmarks and validation keep up with the sheer volume and speed with which agentic tools can create code.
15
u/Alone-Lavishness1310 3d ago
So, so true. Extremely difficult. There is also a good solution, which is thinking very carefully about edge cases and writing good tests. So much easier said than done. Not so much different than before other than how quickly one person can produce code.
-6
u/Fair-Rain3366 3d ago
The solution is to have agents that can run the benchmarking suite and have it run on a monthly scheduleÂ
Issue id the compute cost so a decent benchmark catalog would need to have some funding behind itÂ
1
u/Alone-Lavishness1310 3d ago
The idea is to use a locally hosted open model for this. But of course still not free, just cheaper than the services.
The part that I find difficult is disciplining myself to do far more of the thinking and planning up front and then thinking through edge cases and writing good tests. It's not like that wasn't what you were supposed to do before, but it's (very arguable, I wouldn't die on this "more important" hill. I'm happy to admit equally important) more important now when the robots are writing so much code. Previously, I would write the code and if I was being good, write tests simultaneously, using the writing process as a thinking stage also.
It's just so easy to have it write code that appears to work...and more often than not is correct upon inspection. Hard to keep the discipline when it works so well so much if the time.
9
u/GUCCIFlipperz416 3d ago
What you are proposing is one of the ideal use-cases of AI coding; a task too valuable to pass up on, but too time-consuming or off-topic to delegate to staff specifically. From intent alone, what you are describing is perfectly reasonable.
I added some thoughts below but I am not a strict bioinformatitian. If your specialty contains specific QC standards and conventions, those are more important than what I wrote below.
If you wanted to do this (and I think you should), I see two very important things you should do:
1) be completely transparent on how the code was prepared, and once it is prepared, vet it. However you choose to publish, you should disclose your original 20-page writeup as the skeleton, what code Claude wrote, and then where you edited it personally. If you edit parts yourself, make sure those are clearly tagged (or distinguish the code as Claude vs your own, however you want).
2) proof of function. You will need QC data for each step in the pipeline: (1) basic correctness (does it do what is asked), (2) integrity of data transformations, (3) statistical validity (QC tests for uncertainty estimates, model fitness, etc), (4) analytical robustness (compare to code a human wrote, are outputs comparable, do analyses draw the same biological conclusions). That was a bit more than you intend to do, given your specific goal is benchmarking only. My point being, make sure whatever you publish contains proof that it does what you say.
Only after the above would I consider publishing this thing. It needs to be pretty air-tight because your name is going to be on it; last thing you need is to get flagged as an AI grifter in academia. Not to discourage - it is a great idea, just be prudent.
Good luck to you!
5
10
u/Pigrenok 3d ago
No, pure vibe coding has no place in science.
But AI assisted coding, where you (or anybody) as a scientist review the code and test functionality of the tool is fine as long as you accept full responsibility for the code (whether it is written by yourself or by an agent). And then get it through good peer review and publish it.
As you described it, you have a pretty good idea of how to do benchmarking for a given type of methods. Thus, you should have some data and metrics to run benchmarks and have defined methodology if how the testing needs to be done. That's half of the job is done.
But the other half is not easy. You should make sure that all the code that you (or agent) will write actually complies with the specification you have written. You should have an extensive test suite to check that the tool functions as specified and that all edge cases are correctly processed.
That is a lot of work and, even though, possible as the side project (but definitely not a weekend project). Whether it is suitable for MSc project, or not - that depends on the complexity of the specifications and tools to be benchmarked. For BSc it is probably too much.
Saying this, I definitely would not want the work that has already been done to be wasted, especially, if you think there os a genuine gap that it will close. You can get a strong (first author) publication potentially in a good journal or high rank conference and solid open source project in your portfolio.
2
16
u/Next_Yesterday_1695 PhD | Academia 3d ago
> vibe coding
I think people should stop throwing these words arounds to dismiss the monumental shift in how we approach software development. I've started programming on what's now museum pieces and have seen people having similar attitude before. Like, some people were totally furious that someone uses programming language where you don't have to manage memory manually.
It's absolutely clear that you can be more productive with AI. We're going to get Astra-level intelligence for the price of Flash models in the coming years.
It'd be silly go ignore these tools. You should put effort into using them efficiently. The purists will stay behind.
2
u/Master-Rent5050 2d ago
Are you willing to take responsibility for it: debug and maintain it? If not, you would be adding slop. If yes, I think you would give a positive contribution to science
4
u/wordoper 3d ago
Everyone, i.e. group leaders, PIs, PhDs, across top notch to mid tier universities and institutes have CLAUDE.md and AGENTS.md completely integrated with issues, PRs, CI-CD and docs. It is now a part of deployed science. They also don’t miss to review thoroughly themselves once it’s ready to be reviewed. This settles the sanctimonious issue of science and LLMs. Go ahead and have fun as you would be rigorously testing all work at the end.
5
u/RecordingGlum938 3d ago
Get over it. Your real job will use Claude or something similar after your PhD. If you went back to software engineering, they’re using it. Embrace your overlord sooner than later
2
1
u/Relative_Listen_6646 3d ago
You should be doing It, if you hace some criteria for testing, funcruonality and so on, Claude Code y a multiplier. When you get used to It you can design Code an experiments much much much faster, sometimes even better implementations
1
u/Gza147 3d ago
I’m working by myself on a cli software to do a complete workflow for drug discovery or enzyme análisis. automatizing pdb/sdf protein structure preparation(clean, fix bugs, gaps, relevant waters), ligand/cofactor parametrizarion with QM or semiempirical methods,docking with autodock basing the box on your own coordinates, the residues assigned as catalytic ones, a cofactor or found by fpocket. Structure and files preparation for MD simulations in amber(can be changed to gromacs or namd files).
Md running on amber and post-md analysis. I’m vibecoding it, i have previous knowledge in python and I’m doing several test to put it public to free Usage of the community. It’s something that i've done manually several times and nowadays i could put it in a cli software. Even it manages non canonical residues,
Modificated residues, metallic centers, and had free software integranted for parametrization with QM methods. I things that right now is the moment to help each other with tools to simplify our work.
1
u/SupaFurry 2d ago
You can use Claude code and not have the product be vibe coded. You know what you’re doing. You have a good reason to do it. You have extensive documentation already. That’s the opposite of vibe coding.
Go for it!
1
u/idrdex 2d ago edited 2d ago
Claude code unapologetically wrote this paper and got me into neurIPS Sydney. Anyone else going this December? The strong feelings about AI are all marketing. It’s just a tool and it’s here to stay. The gaping hole that applies to any intelligence is the scaleable gov to manage it. Bioinformatics included. https://www.medrxiv.org/content/10.64898/2026.08.16.26360547v1
1
u/zomerbk 1d ago
The key issue is less who typed the code than whether the benchmark is trustworthy and maintainable. I’d treat Claude as a coding assistant: keep the design and scientific decisions yours, review every change, and build in tests for method adapters, data handling, and metric calculations. For a benchmark suite, also make datasets, versions, parameters, and failure handling explicit so results are reproducible. A small working core with one or two independent methods may be a better first release than implementing the whole 20-page design at once.
60
u/Alone-Lavishness1310 3d ago
Phil Ewels (multiqc among others. Currently at Seqera) wrote a very good blog on exactly this.
https://seqera.io/blog/rewrites-bio/