US Library of Congress set to archive every public tweet ever

Every public tweet ever posted, since March 2006 when Twitter began, will be archived by the US Congress. Whether it be the first tweet by co-founder Jack Dorsey or Conan O'Brien's tweet announcing his surprise following of Sarah Killen, all publicly posted tweets will be included within the archive. 

Twitter and the Library of Congress both announced the decision on their blogs. Twitter boasts of having over 55 million tweets posted on their service each day. The Library of Congress will be available to use the tweets six months after they are posted for "internal library use, for non-commercial research, public display by the library and preservation".

Twitter's general counsel Alex MacGillivray spoke to the BBC regarding the announcement, "I think it shows the tweets are an interesting part of the historical record. It is not something we imagined when we were forming the service."

Twitter in the past and continues to allow its users to instantly publish breaking news and have public comment immediately. Matt Raymond's Library of Congress blog post speculated that the collection of tweets could have some positive effects for research of social behaviour. "I'm no Ph.D., but it boggles my mind to think what we might be able to learn about ourselves and the world around us."

The Library of Congress already stores over 167 terabytes of web-based information with tweets being the latest edition to the large collection of material. 

Report a problem with article
Previous Story

Apple patches Pwn2Own exploit, issues many more bug fixes

Next Story

HP suspected of bribery after Russian raid

32 Comments

Commenting is disabled on this article.

OctiPsy said,
Why the big fuss? Don't want them to archive the tweets, don't use the service.

Or don't make them public. People whine for the sake of whining sometimes. I think this is great, it'll be interesting stuff in a 100 years.

thornz0 said,
...I think this is great, it'll be interesting stuff in a 100 years.

Yes, it will be interesting to see what people were doing at 140 characters or less. Seriously, this is worthless. 99.99% of all tweets aren't worth archiving.

Edited by RangerLG, Apr 15 2010, 3:30pm :

RangerLG said,

Yes, it will be interesting to see what people were doing at 140 characters or less. Seriously, this is worthless. 99.99% of all tweets aren't worth archiving.

And the rest 0.01% is about temporary news that hardly can transcendent time.

I understand that my posts are public and therefore I give up the privacy part. I did put it out there for the public. I never did indend my messages for people from all over to see it though. I have friends from around my area and family from across the country. I live in Canada. I'm not very comfortable with the US storing my posts but I guess I gave that right up when I confirmed my account and started posting. It is an American company afterall. I doubt I'll be back to post more though. I do follow others so I'll be there to keep updated but no more posts from me.

dogmai said,
I understand that my posts are public and therefore I give up the privacy part. I did put it out there for the public. I never did indend my messages for people from all over to see it though. I have friends from around my area and family from across the country. I live in Canada. I'm not very comfortable with the US storing my posts but I guess I gave that right up when I confirmed my account and started posting. It is an American company afterall. I doubt I'll be back to post more though. I do follow others so I'll be there to keep updated but no more posts from me.

..or you could just make the posts private and they won't be indexed or seen outside of who you want..

HeLGeN-X said,
I just opened a twitter account today, and now I am closing it.

You are closing an account because they are indexing it? comon..... you opened a PUBLIC account on the internet!

Auzeras said,
I wounder if they archived my website. its backing it up for free in my opinion

Funny reply. What if you server crashes? You think they will ever sent you the data that was on your site to you..?

Auzeras said,
I wounder if they archived my website. its backing it up for free in my opinion

Even if the LoC didn't, it's likely that Arcive.org's Wayback Machine did. An old site that I did years ago giving information for people about Usenet is there.

protocol7 said,
That's almost as big a waste of time as archiving every comment on Neowin. Including this one.

+1. You beat me to it. Looks like they have got their hands on ample of time and money to waste.

I don't have a Twitter account, but it bothers me that the US Library of Congress stores whatever people outside USA are posting. What if someone wants to post something that is none of USA business? Is this because Twitter servers are on US soil?

What a waste of time. I understand wanting to archive things to preserve them, but twitter has to be one of the worst places to archive. The amount of garbage on there is orders of magnitude higher than actual useful information.

homeboy rocketshoulders said,
What a waste of time. I understand wanting to archive things to preserve them, but twitter has to be one of the worst places to archive. The amount of garbage on there is orders of magnitude higher than actual useful information.
I agree, but on the other hand, it's a direct peek into a lot of users' lives all collected at the same website. I'm also not sure the LoC is looking for "reliable information" alone a la sourced info on Wikipedia, but can accept noise if that's what it takes to have an accurate peek into the world today. There's a lot of that information that is lost in the transition to intentionally documented material, anyway.

Edited by Northgrove, Apr 15 2010, 7:50am :

s0nic69 said,
how long would it take my computer to index all that

IF you had 167 TB of HDD's at your home and the data would be written at 30MB/s average, then it would take 556667 seconds to copy all the data to the HDD's. So that means 155 hours or 6.4 days to finish the job (:

TrOjAn. said,

IF you had 167 TB of HDD's at your home and the data would be written at 30MB/s average, then it would take 556667 seconds to copy all the data to the HDD's. So that means 155 hours or 6.4 days to finish the job (:

That isn't bad at all! I'm going to download the internets.

RedFlow said,

That isn't bad at all! I'm going to download the internets.


rofl
first you shall have to make program that automatically saves 'the internet' for you ;-)

"The Library of Congress already stores over 167 terabytes of web-based information with tweets"
Seems like these guys really want to know everything of everybody o.O

TrOjAn. said,
"The Library of Congress already stores over 167 terabytes of web-based information with tweets"
Seems like these guys really want to know everything of everybody o.O

More accurately they want to do as good a job archiving the internet as possible to give the future generations a deep view into our time.

Frazell Thomas said,

More accurately they want to do as good a job archiving the internet as possible to give the future generations a deep view into our time.

and what about our privacy?

TrOjAn. said,

and what about our privacy?

I'm a big fan of privacy, but it isn't a party in this discussion. You throw privacy out the window when you yell in the middle of the public square (or in this case post a PUBLIC tweet).

Frazell Thomas said,

I'm a big fan of privacy, but it isn't a party in this discussion. You throw privacy out the window when you yell in the middle of the public square (or in this case post a PUBLIC tweet).

I'm going to read Twitters terms of use and see if they are allowed to store my tweets.. i'm having vacation so I can waste some time on this

TrOjAn. said,

I'm going to read Twitters terms of use and see if they are allowed to store my tweets.. i'm having vacation so I can waste some time on this


You really don't understand the concept of publicly posted information, do you? Even if Twitter's TOS didn't allow them to store the tweets (which obviously it does, otherwise, how would people be able to look at your previous ones?), nothing is stopping a third party from archiving them.