henry 发自 凹非寺
Qbit | Public Account QbitAI
I really beg you, there are so many papers that I can’t even see them all, yet the tools for automatically writing papers keep coming out!
Recently, GitHub has released a new AI paper generation tool—
Open Academic Paper Gen.
Don’t rush to leave right away!
In addition to automatically generating a first draft with references, Open Academic Paper Gen has another point you need to pay attention to.
If the full text of the literature is available, it will search for evidence in the full text (emphasis) to verify the cited viewpoints in the paper.
Let’s be honest, this is really practical.
Because in paper writing, often it starts with a research topic, continuously organizing ideas and searching for evidence, and then adjusting one’s judgment based on the literature.
With AI, arguments can help you expand, and evidence can help you retrieve sources.
But the problem is that sometimes the opinions and papers are true, but once the “opinions” given by AI are inserted back into the full text and viewed in context, it becomes clear that the author may not have actually meant that.
So, to solve this problem, Open Academic Paper Gen will further locate the relevant paragraphs after obtaining the full text of the literature, and determine whether the literature can support the corresponding argument.
If there is page number information, it will also indicate the corresponding PDF page number in the comments, making it easy for people to refer back and check.
That is to say, it will not only check whether “this paper exists”, but also verify “whether these words are indeed said by it”.
A multi-agent workflow for generating a preliminary paper draft
Overall, Open Academic Paper Gen is a multi-agent workflow for generating initial drafts of papers. It breaks down the process from topic selection to output generation into nine stages.
Determine the research scope, search for literature, clean and filter, analyze trends, find entry points, create outlines, write, verify, and finally export.
Similar to other AI paper writing tools, you provide it with a research topic, such as “RSI in the Physical World”.
It will extract specific research questions from this topic, then generate corresponding Chinese and English keywords, and search for papers on OpenAlex, roxx, Semantic Scholar, and arXiv.
The literature retrieved will be further deduplicated and scored, and then filtered by the model based on relevance. For papers with high relevance, the search will continue along their references and studies that later cited them.
It’s like catching one article and then further exploring the trail to get more ones, which is actually a bit similar to the CONNCTED PAPERS back in the day.
(Does anyone understand it?)
Next, the collected literature will be organized into an evidence table, summarizing the research questions, methods, data, conclusions, and limitations of each paper.
With these materials, the system can further analyze research trends, and propose possible research angles, research gaps, and hypotheses.
There’s also a quite interesting component called Innovative Diagnosis.
Open Academic Paper Gen evaluates the entry point from several dimensions including issues, methods, data, and perspectives. It identifies the closest existing works in the literature pool and outlines the possible questions that reviewers might raise.
(This is equivalent to having the AI check if others have done it before you think you’re creating a ResNet.)
Specifically, these judgments will focus on the literature retrieved this time. However, the absence of it in the literature pool does not mean that no one has done it worldwide.
Finally, after the research perspective is determined, the system continues to generate an outline and draft each chapter.
After the initial draft is completed, proceed to verify references and conduct a simulated review. Finally, you can export it in Markdown or LaTeX, as well as in reference formats such as RIS and BibTeX.
In terms of specific operation modes, under the default mode, Open Academic Paper Gen will stop at the main stage. It will proceed only after you review, edit, and confirm it.
If you want to test the capabilities of AI research, you can also let it run automatically and choose the fully automatic mode.
Finally, friends who are ready to start running should also pay attention to the model configuration:
Currently, the project supports OpenAI and Zhipu GLM, but several steps such as argument verification, unreferenced statement checking, and simulated review still require an OpenAI API key.
How can I find references in a sentence?
To be honest, automatically searching for literature, writing a first draft, and then having an AI review it is a common combination in many paper writing projects.
Some of them also include reference verification and manual confirmation, trying to minimize random fabrication by the model.
Open Academic Paper Gen In this process, the checking and editing of references are further refined.
After all, adding references to a first draft is one thing, but proving that these references can actually support the main text is another thing.
Specifically, it breaks the verification into three checks.
First, check if the references can be found in the literature pool.
During writing, each reference tag must correspond to a paper that has been collected.
If the model temporarily creates a non-existent reference identifier, the system will mark it so that it can be located and modified easily.
Second step: verify the identity of this literature.
For papers with a DOI, it will first be checked against the crossref records; if not found, it will be confirmed on doi.org.
Title, author, year, and other information do not match, which will trigger the corresponding warning.
Thirdly, check whether the original text supports or does not support this sentence.
At this point, the inspection truly moves from “literature information” to “literature content”.
After obtaining the full text, the system will identify the paragraph that most matches the cited argument, and submit it together with the beginning of the paper to the model to determine whether the original text can support the statements in the main body.
If the full text cannot be obtained, use a summary or excerpt. When page numbers are available, a warning can also indicate the corresponding PDF page number for easy reference later.
Of course, not finding sufficient evidence for the model does not mean that this statement is proven to be wrong.
According to the processing method of Open Academic Paper Gen, when there is insufficient evidence, it can be marked as “cannot be determined”.
If the text is too short or the literature cannot be verified at all, it will be marked as “not verified”, rather than being directly considered passed.
So, overall, compared to “writing the paper in one click”, Open Academic Paper Gen indeed has a very effective approach for organizing references, locating evidence, and checking citations.
Especially when facing the views and arguments written by AI, it often feels like “it makes sense, but I don’t dare to believe it directly”. Being able to find the original text can at least provide an additional way to verify it later.
By the way, this project was created by a personal developer Cat Uncle. In addition to Open Academic Paper Gen, he has also released several other projects such as AI screenwriters and AI research.
Reference link
[1] https://github.com/mmlong818/open-academic-paper-gen