Posting Some Data at Last!

First data file available for download

Posted a new article in the TEI collection which explains how the <said> element works, provides a CSV file for download that extracts all such nodes, and explains what each column in the file contains.

I still need to show some examples or demos of what can be done with this data, but big step forward.

I’d forgotten how much I enjoy the Git workflow:

1. Drop ideas as they come in the issues tracker
2. Start a new branch to work on something
3. Commit code when features work, repeat as needed
4. Create Pull Request from working branch to main
5. Merge PR, delete branch
6. Check issues tracker for idea to work on

What It Took to Get Here

• Clarifications:
  • Clarify said element and provide CSV file
    • Run Xquery in private repo
    • Outline the post and how to make it - [x] Create a data folder just under root, to contain CSV files for download/analysis
    • Create new TEI post which will:
      • pull in xml from private repo
      • tidy the data
      • add a word count column
      • remove raw text column
      • save the data to CSV
      • Display a summary/health-check for file at top of page
        • Histogram of elements without a speaker or an addressee by chapter
      • Provide link for download
      • Explain the data
        • What is a row?
        • What are the columns?
  • Clarify NA vs. NULL values in said.csv
    • Research proper usage:
      • NA means missing data. Flags editing or data collection issues without invalidating the entire row.
      • NULL means the data does not exist. It would invalidate the row and raises errors during assignment.
    • Define usage for project:
      • Use NA when the data is required but missing, e.g. for speaker. Will usually flag that the editing missed or has not gotten to this line.
      • Use NA when the data is optional and missing, e.g. for place. Will not flag an issue with editing, but will drop out nicely when filtering or grouping or summarizing by that column, as opposed to, say, an empty string.
    • Update column descriptions to call out whether NA is normal or not.
    • Add a book column
    • Rename index column to line and move to after page
    • Publish updated file
• Check skimr package for data summary
  • Research usage
  • Implement on the page (used the awesome gt package for layout)
• Enhancements:
  • Removed date prefix from URL of first post in the TEI collection
    • Add RSS feed for TEI collection to nav bar
      • Turned RSS section of nav into a dropdown menu
      • Added a link to each of the existing feeds
• Bugs:
  • Stop rendering the code of conduct file during site builds
    • Moved CODE_OF_CONDUCT.md to .github folder
    • Removed unnecessary yaml header from file

Lessons Learned

skimr

A really cool package to generate summaries about data.

gt

A snazzy, tidyverse-compatible package to generate tables in Rmarkdown. For when a plot is overkill or not as easy to read.