---
name: manage-bibliography
description: >
  Create, merge, and deduplicate BibTeX bibliography files using R packages
  (RefManageR, bibtex). Parse .bib files into structured R objects, merge
  multiple bibliographies with deduplication by DOI or title similarity,
  generate entries from DOI/ISBN/arXiv ID, and export clean sorted .bib files.
  Use when creating a new .bib file for an R Markdown or Quarto project,
  merging bibliographies from multiple collaborators, deduplicating a .bib
  that has grown through copy-paste accumulation, or generating BibTeX entries
  programmatically from DOIs or other identifiers.
license: MIT
allowed-tools: Read Write Edit Bash Grep Glob
metadata:
  author: Philipp Thoss
  version: "1.0"
  domain: citations
  complexity: intermediate
  language: R
  tags: citations, bibliography, bibtex, R, RefManageR
---

# Manage Bibliography

Create, merge, and deduplicate BibTeX bibliography files using R. This skill
covers the full lifecycle of bibliography management: parsing existing .bib
files into structured R objects, generating new entries from identifiers (DOI,
ISBN, arXiv ID), merging multiple bibliographies with intelligent deduplication,
and exporting clean, consistently formatted .bib output.

## When to Use

- Creating a new .bib file for an R Markdown or Quarto project
- Merging bibliographies from multiple collaborators or sources
- Deduplicating a .bib file that has grown through copy-paste accumulation
- Generating BibTeX entries programmatically from DOIs or other identifiers
- Cleaning and standardizing an existing .bib file (consistent keys, sorted fields)

## Inputs

- **Required**: Path to one or more .bib files, or a list of DOIs/ISBNs/arXiv IDs
- **Optional**: Output .bib file path (default: `references.bib`)
- **Optional**: Deduplication strategy (`doi`, `title`, `both`; default: `both`)
- **Optional**: Sort order (`author`, `year`, `key`; default: `key`)
- **Optional**: Key generation pattern (default: `AuthorYear`)

## Procedure

### Step 1: Install and Load Required Packages

```r
required_packages <- c("RefManageR", "bibtex", "stringdist")
missing <- required_packages[!vapply(required_packages, requireNamespace,
                                     logical(1), quietly = TRUE)]
if (length(missing) > 0) install.packages(missing)

library(RefManageR)
```

**Expected:** All packages load without errors.

**On failure:** If RefManageR fails to install, check that `curl` and `xml2` system
libraries are available. On Ubuntu: `sudo apt install libcurl4-openssl-dev libxml2-dev`.

### Step 2: Parse Existing .bib Files

```r
bib <- RefManageR::ReadBib("references.bib", check = FALSE)
message(sprintf("Parsed %d entries from references.bib", length(bib)))

# Inspect structure
print(bib[1:3])

# Access fields programmatically
keys <- names(bib)
years <- vapply(bib, function(x) x$year %||% NA_character_, character(1))
```

**Expected:** A `BibEntry` object containing all entries from the file. Entry count
matches the number of `@article{`, `@book{`, etc. blocks in the file.

**On failure:** If parsing fails, check for unmatched braces or invalid UTF-8 in the
.bib file. Run `bibtex::read.bib()` as a fallback with stricter parsing.

### Step 3: Generate Entries from Identifiers

```r
# From DOI
entry_doi <- RefManageR::GetBibEntryWithDOI("10.1093/bioinformatics/btz848")

# From a vector of DOIs
dois <- c("10.1093/bioinformatics/btz848", "10.1038/s41586-020-2649-2")
entries <- do.call(c, lapply(dois, function(d) {
  tryCatch(
    RefManageR::GetBibEntryWithDOI(d),
    error = function(e) {
      warning(sprintf("Failed to fetch DOI %s: %s", d, e$message))
      NULL
    }
  )
}))
entries <- Filter(Negate(is.null), entries)
```

**Expected:** BibEntry objects with complete metadata (title, author, journal, year,
DOI) for each successfully resolved identifier.

**On failure:** DOI resolution depends on the CrossRef API. If requests fail, check
network connectivity and whether the DOI is valid. Rate limiting may apply for
large batches; add `Sys.sleep(1)` between requests.

### Step 4: Merge Multiple Bibliographies

```r
bib1 <- RefManageR::ReadBib("project_a.bib", check = FALSE)
bib2 <- RefManageR::ReadBib("project_b.bib", check = FALSE)

# Simple merge
merged <- c(bib1, bib2)
message(sprintf("Merged: %d + %d = %d entries (before dedup)",
                length(bib1), length(bib2), length(merged)))
```

**Expected:** A combined BibEntry object containing entries from both files.

Regenerating keys after a merge is a separate job, and RefManageR gives you both
halves: `names(bib)` returns the citation keys and `names(bib) <- new_keys` writes
them back, so `names(bib)[1] <- "newkey"` renames a single entry. Do not assume the
keys you assign survive verbatim — both `names<-` and `c()` push their result
through `make.unique(keys, sep = ":")`, so a second `Smith2020` becomes
`Smith2020:1`; `c()` therefore never drops an entry, but a collision you did not
disambiguate yourself reaches the output .bib as a colon-suffixed key.
Disambiguate first (`Smith2020a`/`Smith2020b`, a coauthor name, or a title word),
then re-read `names(bib)` and keep the old-to-new pairing — every `@key` in an
.Rmd or .qmd body still names the old key.

### Step 5: Deduplicate Entries

```r
deduplicate_bib <- function(bib, method = "both") {
  n_before <- length(bib)
  keys_to_remove <- c()

  for (i in seq_along(bib)) {
    if (names(bib)[i] %in% keys_to_remove) next
    for (j in seq(i + 1, length(bib))) {
      if (j > length(bib)) break
      if (names(bib)[j] %in% keys_to_remove) next

      is_dup <- FALSE
      if (method %in% c("doi", "both")) {
        doi_i <- bib[[i]]$doi %||% ""
        doi_j <- bib[[j]]$doi %||% ""
        if (nzchar(doi_i) && nzchar(doi_j) && tolower(doi_i) == tolower(doi_j)) {
          is_dup <- TRUE
        }
      }
      if (!is_dup && method %in% c("title", "both")) {
        title_i <- tolower(gsub("[^a-z0-9 ]", "", tolower(bib[[i]]$title %||% "")))
        title_j <- tolower(gsub("[^a-z0-9 ]", "", tolower(bib[[j]]$title %||% "")))
        if (nzchar(title_i) && nzchar(title_j)) {
          sim <- 1 - stringdist::stringdist(title_i, title_j, method = "jw")
          if (sim > 0.95) is_dup <- TRUE
        }
      }
      if (is_dup) keys_to_remove <- c(keys_to_remove, names(bib)[j])
    }
  }

  if (length(keys_to_remove) > 0) {
    bib <- bib[!names(bib) %in% keys_to_remove]
  }
  message(sprintf("Deduplication: %d -> %d entries (%d duplicates removed)",
                  n_before, length(bib), n_before - length(bib)))
  bib
}

merged <- deduplicate_bib(merged, method = "both")
```

**Expected:** Duplicate entries removed. Count of removed duplicates printed.

**On failure:** If title comparison is too aggressive (removing non-duplicates), raise
the similarity threshold above 0.95 or switch to `method = "doi"` only.

### Step 6: Sort and Export

```r
# Sort by citation key
sorted_bib <- sort(merged, sorting = "nyt")  # name-year-title

# Export to .bib file
RefManageR::WriteBib(sorted_bib, file = "references.bib", biblatex = FALSE)
message(sprintf("Wrote %d entries to references.bib", length(sorted_bib)))
```

**Expected:** A clean .bib file written to disk with consistent formatting, one entry
per block, sorted alphabetically by citation key.

**On failure:** If WriteBib produces encoding issues, ensure the R session locale
supports UTF-8: `Sys.setlocale("LC_ALL", "en_US.UTF-8")`.

## Validation

- [ ] Output .bib file parses without errors: `RefManageR::ReadBib("references.bib")`
- [ ] Entry count matches expectations (input count minus duplicates)
- [ ] No duplicate DOIs remain: all DOIs in output are unique
- [ ] All entries have a citation key
- [ ] Required fields present per entry type (author, title, year at minimum)
- [ ] File is valid BibTeX (test with `bibtex::read.bib()`)

## Common Pitfalls

- **Encoding issues**: .bib files with Latin-1 accents break UTF-8 parsers. Convert
  encoding first: `iconv -f ISO-8859-1 -t UTF-8 old.bib > new.bib`
- **Unmatched braces**: A single missing `}` silently drops entries. Validate brace
  balance before parsing large files
- **DOI rate limiting**: CrossRef throttles unauthenticated requests. Set a polite
  email with `RefManageR::BibOptions(check.entries = FALSE)` and batch requests
- **Key collisions**: Merging files where both have `Smith2020` does not overwrite
  either entry — `c.BibEntry` routes the combined keys through
  `make.unique(keys, sep = ":")`, so the second becomes `Smith2020:1`. Nothing is
  lost, but a colon-suffixed key you did not choose reaches the output `.bib` and
  no longer matches the `@Smith2020` in your document. Disambiguate before merging
- **LaTeX in titles**: Titles with `{DNA}` or `$\alpha$` need careful handling;
  RefManageR preserves these but downstream tools may strip them

## Related Skills

- `format-citations` - format the bibliography entries into styled citations
- `validate-references` - verify completeness and DOI resolution of .bib entries
- `../reporting/format-apa-report` - generate APA-formatted reports using bibliographies
- `../r-packages/write-vignette` - create package vignettes that cite references


---
Source attribution: pjt222
Original project: https://github.com/pjt222/agent-almanac/blob/cabe7834815765f915822ba1f361aa4623398871/skills/manage-bibliography/SKILL.md
License: MIT
License information: https://opensource.org/license/mit
Original upstream instructions; downloaded from Dave AI.

## Upstream license notice

MIT License

Copyright (c) 2025 Development Guides Contributors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


# Dave AI resource metadata

Creator / publisher: pjt222
Original project: https://github.com/pjt222/agent-almanac/blob/cabe7834815765f915822ba1f361aa4623398871/skills/manage-bibliography/SKILL.md
Attribution basis: Source repository record

Fields of work: Data & Analytics, Education & Research
Roles: Academic Writer, Data Analyst, Researcher
Specialties: Specialty not specified
Tools: R
Country applicability: Country not specified
Classification basis: Source catalog labels
Instruction SHA-256: 2beadf49a642a5e2a1e5deba1b5f77772bbd1c400d0240465671715f365342f6