Friday, February 18, 2011

Committed to seamless genealogy file sharing

DearREADERS,
RootsTech 2011 provided a great deal of answers, but opened up many more questions. Sharing genealogy data is a top concern for Ol' Myrt, and it looks like FamilySearch is considering updating its generic genealogy file system currently known as GEDCOM.
BetterGEDCOM (a grassroots effort) and OpenGen (a top-down developer model with a genealogy database website component) are two entities also in the conversation about improving file sharing. For a listing of GEDCOM Alternatives see the comprehensive list by Tamura Jones.

Hopefully, conversations will continue when it comes to the concept of sharing genealogy database files between researchers, much like this model:

End-user to end-user
Where Researcher A exports a GEDCOM file from his genealogy management program and then directly shares that file with Researcher B via flash drive, CD or some other means. Researcher B opens his genealogy management program and then imports the GEDCOM file.

At  RootsTech, FamilySearch mentioned SoRD as a method of file sharing, but if Ol' Myrt's understanding is correct, that will involve a third party, much like this model:
End-User to website to End-User
Where Researcher A uploads his GEDCOM file to a genealogy website, where Researcher B  downloads from website and imports the GEDCOM file.

Call me crazy, but Ol' Myrt here doesn't like the idea of a website being part of the equation unless it is a disinterested third party like Dropbox, where two researchers agree to share a folder (instead of snail mailing a flash drive) with a generic file that can read by either researcher's genealogy management program seamlessly.

Allowing big-guy genealogy websites like Ancestry.com or even FamilySearch.org to broker data file transfers between individual researchers can prove problematic on at least these levels:
  • How can the end-user determine his data isn't altered perhaps by masks and filters by the website?
  • How can the end user determine his data isn't linked and/or merged by the website to other user-contributed databases?
  • How can the end user ensure that his data isn't retained by the website, when he chooses to take down the file? One might want to do this and upload a more up-to-date file, or completely remove the file with no further interaction with the website.

GEDCOM FILES AREN'T PERFECT
Generic genealogy database files are currently called GEDCOM files. In his posting File Sharing? GEDCOM or what? James Tanner explains "It seems like whomever you need to share information with never seems to have the same program you use. There was a time when Personal Ancestral File (PAF) was pervasive. But with the de-emphasis on the program, there are dozens of different options. Your first door of opportunity in sharing files is to see what formats are supported by your own program."

Unfortunately each genealogy program out there has decided to pick and choose which GEDCOM file parameters it will to embrace. The less-than-marvelous results have been described by genealogy bloggers. It's particularly disheartening that of Legacy Family Tree, Family Tree Maker and RootsMagic only the later can seamlessly export and import it's own GEDCOM file without data loss. See:



While The LDS Church owns the "GEDCOM" name, it hasn't supported the GEDCOM file during the last 14 years. See BetterGEDCOM Wikispaces' History of GEDCOM. According to other RootsTech attendees, FamilySearch says it's GEDCOM revision will begin with the GenTech data model. BetterGEDCOM developers have considered this and a variety of other data models on the BetterGEDCOM Wiki, including:



And the trick is?
Convincing genealogy software producers to adhere to a single "new" file sharing format will be problematic, particularly since they don't currently agree by using all existing GEDCOM specifications. IF they would work this out together, there would be no need for reforms such being explored by BetterGEDCOM. IF end users bring pressure to bear once a good GEDCOM alternative has been developed, THEN perhaps software producers will adopt the plan.

COMMITTED TO SEAMLESS FILE SHARING

Some folks are pleased FamilySearch is weighing in on improving file sharing, thinking that high-profile organization will galvanize support from software producers. Others question FamilySearch commitment based on its history of dropping PAF and ignoring GEDCOM for 14 years. (I don't think FS dropped PAF per se, but rather decided to focus on things they do best like digitizing records.) My commitment is to support the development of seamless end-user to end-user file sharing capability, without requiring a genealogy website to be part of the equation.



FOR FURTHER READING


Happy family tree climbing!
Myrt     :)
DearMYRTLE,
Your friend in genealogy.

9 comments:

  1. Young Myrt,

    > "Call me crazy, but Ol' Myrt here doesn't like the idea of a website being part of the equation unless it is a disinterested third party like Dropbox"

    As you wish; I call you crazy.
    I also agree with you.

    - Tamura

    ReplyDelete
  2. This is a great write-up. Very, very in depth. Tamura also had a pretty insightful post right after Roots Tech finished.

    How do you think the FS API plays in with their new file format?

    ReplyDelete
  3. DearMYRT,

    I refuse to call you Old...or even Ol'

    Amen, sister! Preach it, girl.

    Well put.

    ReplyDelete
  4. I use worldconnect at rootsweb to upload my gedcom. I have upgraded it many times and never have any problem. I haven't tried to cancel it because I want people to see and use it.
    I do not have the download gedcom option checked as some people took a gedcom I sent them early on and put it on family search, errors and all and I now can't correct them. This is one of the unverified DB..
    http://wc.rootsweb.ancestry.com/cgi-bin/igm.cgi?op=GET&db=sbkr2&id=I6347

    My family site is clancywrightgenealogy.org

    ReplyDelete
  5. From Brad in Tampa via his iPhone:

    First, I agree with you. Your conclusions are sound. I can also think of nothing worse than trying to rally genealogists around a file format. Well, except trusting Big Genealogy to get it right. As a TMG guy who recently converted to Mac, sometimes I wish I could just go back to Roots III in DOS and forget all the bells and whistles. But the truth is it must be done. Please keep us posted on opportunities to be heard.

    ReplyDelete
  6. I totally agree with the article, but there are a couple of arguments against transfer via a third party missing.

    Can you trust a third party to keep the secrets in your data?

    And, not less important, if there is no file format - how do you archive your data? Will the third party be around in 10-30 years? How do your grandchildren get access to the data?

    ReplyDelete
  7. Myrtle, there is already a Web standard way of sharing files. It's called peer-to-peer networking, made famous my Napster. But there are many others, such as gnutella. The problem is restricting who can download. Then of course there are all kinds of alternative, such as file transfers via chat and email clients. I can see SORD gaining success among less tech savvy people (not all of whom are older people). It will have a limited life, as kids today will be too tech savvy to be trapped into using some arcane third party interface, when they are older.

    ReplyDelete
  8. With an updated file standard, the following would be possible:

    1. Find information, source, and images about your ancestor.
    2. Click a link to have everything including a source citation imported into your genealogy software.
    3. Smile, knowing that you don't have to re-key data or spend time citing the source.


    In a survey conducted in 2009, 93% of survey participants wanted such a feature:

    http://www.thinkgenealogy.com/2011/02/10/a-better-way-to-cite-online-sourcesreprise/

    ReplyDelete
  9. There needs to be an updated standard, simply to reflect the more "stuff" computers can do now. But just because FamilySearch does one, doesn't mean there can't be others. Once you have the code written that can import/export information to a certain format, it isn't that much more work to implement another format.

    Another thing to think about is if it is possible for you to seamlessly move from app to the next, now you won't have that ugly choice of "starting over" in a new app, or updating your current one. Which way do you figure the software vendor is going to want you to do? What is the motivation for the vendor to export to this new format (import, yes. Export?)? Genealogy is faster and faster becoming monetized (it is a little distressing!), and if it might cost them a sale...

    ReplyDelete