September 25, 2010

How to use EURO-DNA-CALC with Family Finder (FTDNA) autosomal data

Thanks to the people who offered their help!

You can use the existing EURO-DNA-CALC with Family Finder data. There are only 63 SNPs in common between the dataset used in EURO-DNA-CALC and the chip used by Family Finder, so it will be interesting to see what kinds of results will turn up (*)

Here are the instructions on how to use Family Finder data.
  1. Download the zip file of EURO-DNA-CALC and extract its contents into a directory. There is a Readme file which you will use, but first you must convert your data.
  2. You autosomal Family Finder data has a csv.gz extension, i.e., it is a comma-delimited GZIP-compressed file. You should use an suitable program (e.g., Winrar or Winzip) to extract the csv file into the same directory as in step #1.
  3. Open the csv file in any text editor (Word or Wordpad should work fine).
  4. Remove the header (RSID,CHROMOSOME,POSITION,RESULT) at the top of the file
  5. Replace all quotes (") with nothing.
  6. Replace all commas (,) with tabs.
  7. Replace all missing value characters (-) with the character m.
  8. Save the file in the same directory as 23andme.txt. You've just converted your Family Finder data into a format that mimics that of 23andme.
  9. Follow the instructions in the Readme file exactly as if you had 23andme data.
  10. Feel free to e-mail me with your results, as I enjoy hearing from people who've used this tool.
(*) There are 192 and 163 SNPs in common between the results of 23andMe and deCODEme and the Price et al. dataset used by EURO-DNA-CALC, so it is expected that the accuracy of the estimate with Family Finder will be reduced. All three companies test multi-100K SNPs, but only a limited number of them coincide with the 300 AIMs selected in the Price et al. publication on which EURO-DNA-CALC is based.

25 comments:

  1. Thanks for tweaking this tool for us FTDNA people!

    ReplyDelete
  2. Hi Dienekes. You might want to explain to the less computer handy how you do a "replace with". Andrew

    ReplyDelete
  3. Dienekes have you done 23andme? And if yes what was your experience and would you recommend it?

    ReplyDelete
  4. I haven't done 23andMe for reasons outlined here:

    http://dienekes.blogspot.com/2008/09/why-i-wont-be-testing-with-23andme-yet.html

    Since I wrote that, 23andMe have added some fine-scale ancestry analysis, so it is a better idea to test with them now than it was back then. But my known ancestry is rather uneventful, so I'm still not convinced that I would get much bang for my buck.

    With that said, everyone is free to evaluate their attitude towards this and any other test and decide based on their own reasons for wanting to do it.

    ReplyDelete
  5. But my known ancestry is rather uneventful, so I'm still not convinced that I would get much bang for my buck.

    Do the Orthodox Churches have centuries-long records of their every member as in Catholic and Protestant Churches?

    ReplyDelete
  6. My Arcadian sister-in-law has records on her paternal side that go to the 16th c. So, even if the churches as such don't do this, many landholding families did.
    I'm quite surprised that our hero has not joined 23andme...I keep waiting for him to turn up in my Relative Finder. Apparently, my closest relative in the database is an E1b1b1a2*/X1a1 male. I guess I'll just have to settle for dreaming it's George Stephanopoulos. Damn!

    ReplyDelete
  7. Spy, the stimulus behind my question was that people of European Catholic and European Protestant backgrounds usually have centuries-long information about their genealogy in church records, so they don't feel so much need for genetic testing. But as far as I know (I am not sure, correct me if I am wrong), people of Orthodox (including the Greek Orthodox Church) and Oriental Orthodox backgrounds don't have centuries-long information about their genealogy due to the nonexistency of a tradition of keeping the records of their every member in these Churches (as I said, I am not sure of this, so correct me if I am wrong), so it must be impossible for these people to know their genealogy beyond a few generations except for a small privileged minority among them (unfortunately, the same is true for Muslims). So I was surprised at Dieneke's lack of interest in his genetic past.

    ReplyDelete
  8. Genealogy beyond a number of generations is of only historical interest, as most of one's remote ancestors have contributed no DNA to them (a consequence of the fact that your DNA is 3 billion letters long and your ancestors is a few trillion letters long and came down to you in chunks across the generations; some chunks get dropped).

    If my known ancestors were from Constantinople or Alexandria, or some cosmopolitan place, or if they lived in a country (e.g., Italy) where admixture between Greeks and non-Greeks was frequent, then genetics might turn up something interesting. But, they were all Greek Orthodox from two of the most out-of-the-way places in the Ottoman Empire that you could find in Europe and Asia Minor.

    Anyway, I'm not against the idea of testing, but I think I'd feel like a fool if I paid a few hundred $ to find out something I already know. As prices drop and features pile up, I'm sure I will eventually reconsider.

    ReplyDelete
  9. As prices drop and features pile up, I'm sure I will eventually reconsider.

    This is exactly what I think about genetic testing. I haven't tested yet, but I'm sure I will someday test. As a Turk whose known ancestors (none of them are from before the 19th century) were all Sunni Muslim Turks from Europe (the Balkans to be more specific) and Asia Minor (50-50 for both regions), I am probably more interested in genetic testing than you.

    ReplyDelete
  10. Dienekes,
    Since your 2008 post on 23andMe, they have put a lot of work into their ancestry analysis. Although this type of "relative finding" is still in its infancy, I think it is a fascinating exercise. It is the kind of thing where the more time and effort you put into it, the more you get out of it. I highly recommend it for anyone who has an interest in this area.
    (Wait for one of their frequent sales though!)

    ReplyDelete
  11. Onur, you could probably find out a lot more about your family's history by finding out what ancestral village they were from than from 23andme.

    In the Balkans, people tended to stay with their village, or set of villages. For example, in Kozani in Northern Greece, villages were Vlach, Greek Orthodox, and Turkish. Additionally, certain villages were known to be Muslim, but comprised of converted Greeks.

    Even though the Balkans lack extensive genealogical records, grave yards tend to go back several hundreds of years. Surnames are very specific, as are the conventions of passing down of first names. Certainly, any villager can tell you who they are related to going back at least five generations.

    ReplyDelete
  12. Would anybody really pay to know that their seventh cousin is so and so? When I found out that 23andme was marketing a cousin database, with the ability to interact with said cousins, it was a WOW!-people-will-buy-anything moment for me.

    I have a somewhat unusual last name and occasionally run into strangers with this name. It's always pleasant to say hello, but really not possible or very meaningful to keep track of it all.

    ReplyDelete
  13. Onur, you could probably find out a lot more about your family's history by finding out what ancestral village they were from than from 23andme.

    I already know my father's and maternal grandfather's villages (all I know about my maternal grandmother's ancestors' now-lost village is that it was in Thessaloniki), but the information they provide is very limited.

    Surnames are very specific

    Muslims didn't have surnames until the early 20th century.

    as are the conventions of passing down of first names

    I think this is true to some extent especially in the old generations of Turks.

    Certainly, any villager can tell you who they are related to going back at least five generations.

    And usually not much more than that at most. You cannot find centuries-long genealogies as in Catholic and Protestan Europe in Muslim lands except for a very tiny privileged minority.

    grave yards tend to go back several hundreds of years

    Maybe that may provide some interesting information. But first I have to learn reading Turkish in the Arabic alphabet for the Ottoman-era epitaphs.

    ReplyDelete
  14. In the Balkans, people tended to stay with their village, or set of villages.

    Not different from Asia Minor or rest of the world before modernization.

    ReplyDelete
  15. Well, if I think about it....

    I spend roughly 1000 Euro for genetic testing so far and found nothing I did not know already. :-P

    But I hope to help science by providing my data and that my data becomes more usefull for me, if more people test and better analysis will be invented.
    Also, its pretty interesting to read about haplogroups an KNOW wich is your own. ;-)

    What I did so far:
    Y-DNA at 67 Markers + several SNP checks on new SNP (all negative)

    mtDNA (Full genome sequencing)

    mtDNA of my father mother at HVR1+2 level

    500K autosomal SNP "Family Finder"/"Population Finder"

    atm I am really tired on testing.
    But I think the most interesting is autosomal DNA. I mean... after all its autosomal DNA that is responsable for hwo you look, how your character is, what you have talent for etc.

    I mean, the bottle is the mtDNA. The writing on it is your y-DNA. But what is inside the bottle, is actually the autosomal DNA. And isnt that the important thing?

    If there is milk in a coke bottle wichs writing says "beer"? Its milk, thats what counts, not that there was coke or beer inside it, some 5000 years ago.

    ReplyDelete
  16. "I know about my maternal grandmother's ancestors' now-lost village is that it was in Thessaloniki"

    I think I've mentioned Mark Mazower's book to you, "Salonika, City of Ghosts."

    There were many villages surrounding Thessaloniki. People, Christians, Muslims and Jews, often lived part time in the city and part time in their surrounding village. Mazower's book describes this village-city Salonika way of life. It's the best reference for Salonika that I've come across and is a good starting point to understand the rich history of this vibrant city.

    ReplyDelete
  17. I have a somewhat unusual last name

    You mean your last name from your husband? Greek last names are generally long and sound somewhat strange to Western ears (I think they are cool).

    ReplyDelete
  18. Sorry, my maternal grandmother's ancestors' village was in Drama, somehow it dwelled in my memory as a village in Thessaloniki. I've made a little search for it and found that it was a village with a Turkish majority and all of the rest being Bulgarians. During the Ottoman times the village had a non-Turkish name, likely Bulgarian or even Ancient/Medieval Greek (it had a Greek-sounding name). It was founded before the Ottoman times and probably as a Bulgarian village. The village still exists, but with an entirely Christian population since the 1923 population exchange, and its name was changed to a Modern Greek one in 1927.

    ReplyDelete
  19. somehow it dwelled in my memory as a village in Thessaloniki

    That is very normal as Drama and Serres were parts of the Salonica (=Thessaloniki) province (=vilayet) during the Ottoman times:

    http://en.wikipedia.org/wiki/Salonica_Province,_Ottoman_Empire

    I mention Serres, as my maternal grandmother's ancestors' village was a part of the Serres sanjak, and was joined to Drama only after the region was conquered by Greece during the Balkan Wars.

    ReplyDelete
  20. conquered by Greece

    1. liberated by Greece
    2. you continue flaunting the multiple posting rule.

    ReplyDelete
  21. I chose "conquer" as it is a neutral word. Of course, you are free to call conquest of a region (I mean the sub-region in Serres/Drama that included my maternal grandmother's ancestors' village) that was predominantly inhabited by Bulgarians, Turks and Vlachs by Greece as liberation.

    ReplyDelete
  22. Onur, as you know, there's quite a bit written, both online and off, about both the prefecture and town of Drama. It's a famous place.

    Most of the Turkish population left with the exchange in the 20's. Northern Greece is historically a complex web, both religiously and ethnically. Again, I'd highly recommend Mazower's book. Even though it focuses on Thessaloniki, it paints a picture of cultural complexity that is hard to grasp today.

    I had a bit of a look online, and as with Upper Greek Macedonia (Kozani), starting in the 15th century, the Orthodox appear to have taken refuge in mountainous villages, while the Ottomans were able to take advantage of the agricultural plains. There seemed to have been a lot of deal making between the traditional inhabitants of the plains and the Ottoman Beys. Some Orthodox mountain villages obtained pledges of protection from the Beys. Some "Turkish" villages of the plains were comprised of people who had once been Orthodox.

    Onur, I'm avoiding using the word Turkish, Bulgarian, Macedonian or Greek here. Our 21st century conception of those national names distorts the understanding of the interwoven web of the Southern Balkans of two or three hundred years ago.

    In my experience of Kozani, there are a surprising number of old people who remember Ottoman times. They're dying quickly though, so time is of the essence in listening to their stories. The prefecture of Drama is probably similar to Kozani in this respect.

    It's surprising what the old people (Orthodox) will tell you and how complex was their relationship with the non-Orthodox.

    So, in short Onur, I think you could find out a lot about your history by doing some research up front. Then go to Drama and talk, with subtlety and respect, to some of the old people.

    Drama also has some excellent museums, including an archaeological museum and a folk museum.

    ReplyDelete
  23. It's a famous place.

    I know.

    I had a bit of a look online, and as with Upper Greek Macedonia (Kozani), starting in the 15th century, the Orthodox appear to have taken refuge in mountainous villages, while the Ottomans were able to take advantage of the agricultural plains. There seemed to have been a lot of deal making between the traditional inhabitants of the plains and the Ottoman Beys. Some Orthodox mountain villages obtained pledges of protection from the Beys. Some "Turkish" villages of the plains were comprised of people who had once been Orthodox.

    Onur, I'm avoiding using the word Turkish, Bulgarian, Macedonian or Greek here. Our 21st century conception of those national names distorts the understanding of the interwoven web of the Southern Balkans of two or three hundred years ago.


    I am using those "ethnic names" in a loose sense.

    So, in short Onur, I think you could find out a lot about your history by doing some research up front. Then go to Drama and talk, with subtlety and respect, to some of the old people.

    That would be interesting indeed. But I doubt that even the oldest people know anyhing useful and reliable about the pre-19th century (and even the early 19th century) times.

    ReplyDelete
  24. It's up to you Onur.

    I found this book that mentions Serres, starting on page 197: "The Ottomans and the Balkans: a discussion of historiography" by Fikret Adanir and Suraiya Faroqhi."

    Serres must have been a fascinating place.

    It's been scanned into Google, so it's easy to find.

    ReplyDelete
  25. Disambiguation for ""ethnic names"": "ethnicity names"

    ReplyDelete

Stay on topic. Be polite. Use facts and arguments. Be Brief. Do not post back to back comments in the same thread, unless you absolutely have to. Don't quote excessively. Google before you ask.