Perl Does the Heavy Lifting

by Dietrich Schmitz


By metrics of late, I am an old dinosaur. You see, I cling to technology viewed by some as either falling out of use or just not good enough by their measure.


My eyebrow furrows as I read stories on programming trends and I chuckle at others with what is passed off as being genuinely novel.


The evangelicals proclaim: "A new scripting language with full Object-Oriented Programming (OOP) support for Javascript" (sigh) -- or, "We all should be using node.js." -- #facepalm. Which is not to say that my personal views should preclude the possibility that there are genuinely new programming paradigms at play, allowing each language to grow its own ecosystem. Indeed, they exist. It's just that many languages have been influenced by Perl, including Python, PHP, Ruby, ECMAScript, Javascript, and Windows PowerShell. Use the right tool for the right job but don't throw the baby out with the bathwater.

One Language for All

I do hate web programming -- mostly because it often requires knowing several languages to cobble the 'moving parts' together into a functioning website. Somehow, I feel it is a kludge. There are, of course, Model View Controller (MVC) frameworks which ease the burden of programming but still one must switch hats from html to templates to css to sql to scripting languages, both server- and client-side details. The level of effort required to get up a website varies according to requirements, scale and implementation details.

The goal of language frameworks like Google Web Toolkit (GWT) attempts to unify all coding into one language. GWT uses Java but in actuality doesn't conform to the GPLv2 with certain proprietary bindings either changed or omitted by Google. Follow-on efforts like Go and Dart are off to a slow start and their adoption depends on a Google-Apps for Business and Google-Apps-Engine centric approach. That's fine if you don't care that you've been 'roped in' by Google and want to focus on one language for everything, but if not, then such language implementations simply won't do. Enter PHP, Perl, Python with solutions all their own. What makes one better than the other can be very subjective and the degree of hair-splitting rivals the religious experience.

There are some good reasons why Perl has lasted just a little over 25 years. The struggle (or failure depending on your view) to get Perl 6 off the ground is perhaps indicative of just how dependent we are on Perl 5.x and how comprehensive a code base it has become, which continues to thrive as represented by CPAN, the Comprehensive Perl Archive Network. The name is quite appropriate and descriptive of a body of work done by thousands over the years since Perl's inception. So, to some extent, when discussion turns to 'one language for all', I prefer to think it has already been done -- Perl facilitates writing everything in a dynamic scripting application environment.

Maintainability

Undertaking any software development project, however large, requires careful planning and consideration of key factors that can make the difference between overall success and failure. chromatic at ModernPerlBooks.com wrote a story entitled Mrs. Feynman's Advice on Programming Language Popularity Contests, and distills a concise list of his project management concerns:

If we actually talked to people about what they wanted, we'd find out that they care about things like:
How easy is it to hire and/or train people in a language?
What are the deployment concerns for the language?
What are the security and support channels like?
How many programmers do you need to accomplish a task?
Is the language and its ecosystem suitable for one task or another?
Will a cancellation of the language by its primary vendor or abandonment by its single author or forking of its community have any detrimental effect on recruiting and retaining?
Does the language and its ecosystem support the desired platforms?
Will choosing this language solve more problems than it creates?

Popularity and Expressiveness

Every software development project requires finding talent. And, the degree to which programmers specialize in one language verses another varies and dictates the level to which resources are available in a given region to hire. Redmonk.com have put together their own annual analysis of programming languages and with two stories, we see to what extent the level of popularity and design integrate with two graphs shown directly below:

Redmonk Programming Language Rankings: 2013

Median-Weighted Programming Language Expressiveness

So, it becomes readily apparent that Perl is still 'in the game' as it were -- holding its own quite well despite its aged status. That speaks volumes not just about its flexibility and durability, but its enduring capabilities and true reliability. It is truly industrial strength and up to meeting the demands of small- to enterprise-scale projects.

Platform Support

Perl runs across multiple hardware platforms, including Windows, OSX, Linux, BSD Unix, IBM's AIX and other Unix dialects. It is pretty much a given that the majority of Linux Distributions will have Perl installed by default.

Vendor Support

Third party Vendor support for Perl remains strong with ongoing contributions coming from ActiveState and Strawberry Perl.

Industrial Strength Uses and Applications

So where is Perl in use? It's not just for the web.

Perl is being and has been used in places you might not have guessed -- such as Computer Chip Design, Bioinformatics, the Human Genome Project, Systems Integration, Data Warehouse ETL, EDI, and the popular Internet LAMP stack to name just a few places.

Major applications written in Perl include WebGUI, cPanel, RT, Bugzilla, TWiki, and Movable Type [Edit: Marco Vittorini was good enough to mention in a comment that DuckDuckGo's engine is written in Perl].

Websites that make use of Perl extensively include familiar names like Amazon, bbc.co.uk, Priceline.com, ImDb, Ticketmaster.com, Livejournal and Slashdot (/.). Recently, even Greg Kroah-Hartman has found Perl useful at the Linux Foundation.

Frameworks and Third-Party Tools

All of the above notwithstanding, while developers will write one-off scripts larger scale projects of increasing complexity need the leverage of pre-programmed routines referred to as 'Frameworks'. Third-Party Perl Frameworks include, Catalyst, Mojolicious, Dancer, Moose, in addition to thousands of other pre-made, tested, certified modules at CPAN. There isn't any shortage of bolt-on parts from which to resource needed functionality. This saves much man-hour time expense and reliability of CPAN's rigid testing certification process ensures bugs will be minimized in the course of program development.

Conclusion

No one programming language is a panacea. That isn't the point here. But one should not forsake a language by virtue of its age. Perl continues to broaden and grow, anneal, case harden like a fine steel Ginzu knife. Regardless of rumors of its demise, Perl remains 'alive and well', stolid, ready to serve those who avail themselves to it for many years to come.

Perl does the heavy lifting. -- Dietrich

Related articles

• Web based debugging article and the Perl Maven over 3000
• the perl 5.10 lexical topic
• Installing new Perl Modules using strawberry Perl access denied issue
• perl hacks: Perl Search Revisited

Read More

U.S. Economic Collapse is Unavoidable

by Dietrich Schmitz

I feel so strongly about what is happening in the U.S. that it is my moral obligation to amplify a message that needs to reach the general public concerning the grave condition of our economy.  Thus I am reprinting a story written by Paul Craig Roberts.

The fact is, that if nothing can be done to avoid the unavoidable, you should at least be 'awoken' and become informed of what is around the corner headed our way.

Would you believe a Man with impeccable credentials if he told you the U.S. economy is heading to a complete collapse?  Then you owe it to yourself to read his story.  Here it is:

The Real Crisis Is Not The Government Shutdown

by Paul Craig Roberts 

October 2, 2013

The inability of the media and politicians to focus on the real issues never ceases to amaze.

The real crisis is not the “debt ceiling crisis.” The government shutdown is merely a result of the Republicans using the debt limit ceiling to attempt to block the implementation of Obamacare. If the shutdown persists and becomes a problem, Obama has enough power under the various “war on terror” rulings to declare a national emergency and raise the debt ceiling by executive order. An executive branch that has the power to inter citizens indefinitely and to murder them without due process of law, can certainly set aside a ceiling on debt that jeopardizes the government.

The real crisis is that jobs offshoring by US corporations has permanently lowered US tax revenues by shifting what would have been consumer income, US GDP, and tax base to China, India, and other countries where wages and the cost of living are relatively low. On the spending side, twelve years of wars have inflated annual expenditures. The consequence is a wide deficit gap between revenues and expenditures.

Under the present circumstances, the deficit is too large to be closed. The Federal Reserve covers the deficit by printing $1,000 billion annually with which to purchase Treasury debt and mortgage-backed financial instruments. The use of the printing press on such a large scale undermines the US dollar’s role as reserve currency, the basis for US power. Raising the debt limit simply allows the real crisis to continue. More money will be printed with which to purchase more new debt issues needed to close the gap between revenues and expenditures.

The supply of dollars or dollar denominated assets in foreign hands is vast. (The Social Security system’s large surplus accumulated over a quarter century was borrowed by the Treasury and spent. In its place are non-marketable Treasury IOUs. Consequently, Social Security is one of the largest creditors to the US government.)

If foreigners lose confidence in the dollar, the drop in the dollar’s exchange value would mean high inflation and the Federal Reserve’s loss of control over interest rates. It is possible that a drop in the dollar’s exchange value could initiate hyperinflation in the US.

The real crisis is the absence of intelligence among economists and policymakers who told us for 20 years not to worry about the offshoring of US jobs, because we were going to have a “New Economy” with better jobs.

As I report each month, not a single one of these “New Economy” jobs has appeared in the payroll jobs statistics or in the Labor Department’s projections of future jobs. Economists and policymakers simply gave away a good chunk of the US economy in order to enhance corporate profits. One result has been to create in the US the worst distribution of income of all developed countries and of many undeveloped ones.

In the scheme of things, the enhanced profits are a short-run thing, because by halting the growth in consumer income, jobs offshoring has destroyed the US consumer market. As I noted in a recent column, on September 19 the New York Times reported what I have reported for years: that US median family income has not increased for a quarter of a century. The lack of consumer income growth is why 5 years of massive monetary and fiscal stimulus have not brought economic recovery.

The real crisis cannot be addressed unless the jobs are brought back home and the wars are stopped. As powerful organized interests oppose any such measures, Congress will pass a new debt ceiling and the real crisis will continue.

Do you hear any mention of the real crisis in the media? Today I was on an international TV program for 25 minutes with the chief financial editor of one of England’s major newspapers. Little doubt but that he was a good-hearted and intelligent fellow, but he had no capability of thinking outside the box. He was unable to comprehend my explanations, and resorted to regurgitations of the media’s ignorance or subservience to Washington’s propaganda.

Among his regurgitations was the “solution” of cutting Social Security. The chief financial editor of a major UK newspaper did not know that for the past quarter of a century Social Security revenues exceeded Social Security payments, and that the Treasury spent the surplus to fund the annual operating expenses of the government, issuing non-marketable IOUs to the Social Security Trust Funds.

The chief financial editor also did not comprehend that cutting Social Security payments also cuts consumer spending or aggregate demand, and sends the economy down further, thus magnifying the deficit/debt problem.

Because of the serious decline in the US economy caused by jobs offshoring and financial deregulation, Social Security no longer adds to its surplus. Social Security payments need the supplement to the annual payroll revenues of repayments by the Treasury of the borrowed funds.

The only reasons that Social Security is in trouble is that jobs offshoring and wars have constrained the US Treasury’s ability to make good on its debts except by having the Federal Reserve print money. Every job that is sent abroad does not contribute payroll taxes to Social Security and Medicare.

Insouciant American economists say that manufacturing is an outmoded source of employment, but Chinese manufacturing employment is almost equal to the total US labor force in all occupations, including waitresses and bartenders and hospital orderlies. China’s economy is growing at a rate of 7.5% in real terms, while Western economies cannot move forward and some are regressing.

In order to appease Wall Street, the most corrupt institution in human history, and to prevent Wall Street-financed takeovers of their corporations, executives destroyed the American consumer market by offshoring American incomes in order to enhance profits by substituting cheap foreign labor for US labor.

In my opinion, the US economy is not salvageable in its present form. The economy is running out of water resources. The supply that remains is being decimated by fracking. The soil is depleted by glysophate, a requirement of GMO agriculture. The external costs of production are rising (the costs that the corporations impose on the environment and third parties) and possibly exceed the value of the increase in corporate output. Economists are incapable of independent thought, and elected representatives are dependent on the private interests that finance their campaigns.

It is difficult to imagine a more discouraging situation.

At this time, collapse seems the most likely forecast.

Perhaps out of the ruins, a new, intelligent beginning might occur.

If there are any leaders.

END

Related articles

• Dr. Paul Craig Roberts: Terrifying US Collapse Ahead !
• Why the Debt Ceiling is a Distraction
• JIM ROGERS on U.S. GOVERNMENT SHUTDOWN - U.S. NOT on Brink of DEFAULT & will PRINT more MONEY.
• David Stockman - The Crisis Today Is Far Worse Than 1981- This Financial Collapse Will Be Catastrophic

Read More

Ubuntu Saucy Salamander Smart Scopes is Dumb

by Dietrich Schmitz

I think I've seen just about everything now.  Just when I thought Canonical couldn't pull another boner move like the Amazon 'Lens' privacy issue, they continue head long into Smart Scopes, a full specification fleshed out for Ubuntu 13.04 and beyond.

I think you'll find the picture below showing Smart Scopes in action descriptive:

Ubuntu 13.10 Saucy Salamander 'Smart Scopes' Feature

My reaction is one of confusion.  And that is from someone who has been around computers for years.

What were they thinking?  Smart?  I say: Dumb!  Very Dumb!  It's a jumbled mess.

The obvious question to me is:  Why should I alter my habit of using a browser which is self-contained and optimized for doing on-line searches, shopping over SSL in favor of this confusing interface?

The answer is: I shouldn't abandon the browser for this abomination.

Do I want my search activity mediated by Canonical Ltd on their web servers?: No way.  I don't care if it's tunneled over SSL.  I'll stick with what works reliably: a browser.

Thankfully, Ubuntu has global toggles to turn this and other features off.  I wouldn't want it and I would suspect the majority of Ubuntu users, new and old, will share the same reaction.

There are some features in Ubuntu 13.10 that might be of interest but this certainly isn't one of them.

Does the Canonical Development Team even do any 'User Acceptance Testing'?

It would seem to me that they'd get an ear-full of negative feedback on this one.  

No, It's deja-vu all over again I am afraid.  Someone pushes a feature request from the top (cough Mark Shuttleworth), and it gets written regardless of whether it even makes sense to do so.

That appears to be how Ubuntu development is governed, with some favoritism mixed in.  Unlikely there is any consensus vote taken for feature acceptance -- just put in -- no questions asked.  I've written before that Ubuntu community are chumps on the outside looking in.  Had they really any control over their 'own' community Distro, would Smart Scopes ever have seen the light of day?  I think not.

It's just more affirmation that the Ubuntu community should seriously give consideration to forking Ubuntu to regain control and rebuild what was once a great product.

Mir, a new 'one of a kind' Display server technology, so far will only be adopted and used in Ubuntu and not by any other Distribution.  Plans announced in June 2013 to add Mir to 13.10 (without an X fallback option) were postponed on October 1, 2013 and so won't be making their debut until version 14.04 at the earliest.  

As mentioned, other Derivative Ubuntu development teams have steered away from supporting Mir in favor of staying for the 13.10 cycle with X.org server technology but a view to adopting Wayland in the long term.

Ubuntu continues on a course which is questionable and with each release cycle a new round of feature enhancements put its future into question.

-- Dietrich

Related articles

• Failed Funding Effort Blunts Canonical's Ubuntu Edge
• Fact sheet: Ubuntu 13.10 (Saucy Salamander)
• Ubuntu 13.10 Beta Final Has Been Made Available For Download

Read More

Linux Mint 15 "Olivia" Xfce Edition Approaches Perfection

by Dietrich Schmitz

I don't heap high praise on a Linux Distribution very often.  In fact, I have taken aim at various camps' Distros with regularity to the point of seeming rather harsh. 

But, that criticism isn't undeserved.  No, too often I see 'me too', 'me too', 'me too' cookie cutter mania.

When one of the major Linux Distributions goes into general release there is an ensuing outbreak 'like a bad rash' of clones which surface within hours of the announcement on Distrowatch.com.

You've seen it and often enough to realize that okay 'maybe' some of them might have some redeeming value, but many are just being 'copy cats'.  FOSS activists will argue for variety.  I argue that too many Distros 'confuse' and send a negative message in terms of perceived quality and consistency.  I have written about standardization here and elsewhere, so you know where I stand on that count.

Yesterday, I was curious about Ubuntu's impending release of 13.10 Saucy Salamander enough to download the Lubuntu 13.10 final beta2 derivative release for a quick evaluation.

It was driven primarily by my interest to see how the newest kernel 3.11.x performs.  Much to my disappointment I found directly after install that Chrome stable flash video was laggy.  I had just reinstalled over the top of Lubuntu 13.04 which has garnered a lot of my respect as being a very lightweight but complete Distribution quite suitable for Netbooks such as my Acer Aspire One D260 with 2GB of ram.  So, I was 'miffed' at the discovery of this.

Okay, this is the final beta2.  I realize that -- don't go firing off a comment filled with invectives just yet.  Hear me out.

With 'Axel' installed, I quickly downloaded the Xubuntu 13.10 beta2, 'dd' copied it to my SanDisk 16GB pen drive, booted up and installed, again over the top.

A half hour later I was up on Xubuntu 13.10 beta2.  I dispatched directly to the Chrome stable flash test with the same website's youtube video.  (Sigh)

Same laggy video.  I check my chrome://gpu settings and see 'all green':

So, I know hardware gpu assisted graphic acceleration is working fine with the supported Intel GMA3150.

What now?  Taps fingers....Okay, I am the proverbial glutton for punishment.  One more time, I went to the Kubuntu website, downloaded their 13.10 beta2 iso, slapped it on the pen drive and installed.  A half hour later,  I was on-line with the same Chrome version, same flash youtube video.  Verdict: Video was laggy.

#Facepalm

That's another two hours of my life I'll never get back again, I thought.

That's really odd even if these are beta Distros.  My instincts tell me now to find another Distro using the same (approximately) kernel as Lubuntu 13.04 that has been in production for some time.

So, I go to Distrowatch.com and stare at the trending list, noting the usual stack of top 10 contenders, and there at the top is Mint.  It occurred to me that I really hadn't tried Mint in quite a while and realized that they have an Xfce edition in their line-up which I directly downloaded, again putting on the pen drive, and proceeded to perform a clean install of Linux Mint 15 "Olivia" Xfce edition.  

Linux Mint 15, released in July 2013, includes a 3.8.x kernel.  Again I tested with Chrome stable the same youtube video -- this time no laggy video.  Huh.

Is this a kernel regression I thought?  Sure seems like it.

So, that really isn't the point here.  My serendipity led me back to a Distro which has always been in the top 5 on Distrowatch for a long time.  And I am most impressed with what Clem and the Linux Mint Developer Team have done.  Clearly, Mint is at the top of today's list for good reason.

The level of professionalism, fit, finish is evident from the moment you boot to login to Desktop presentation.  The Mint line-up of Distributions are arguably a 'cut above' the rest.

Linux Mint 15 "Olivia" Xfce Desktop with Mint-Minimal Icon Theme by +Paulo Silva 

The Xfce edition is as complete a Distribution as anyone can expect yet still has a level of conservative lightweight memory use going in its favor.

This isn't a Distribution review -- it's a commentary on what I see across the landscape and how truly only a small handful of Distros are worth their salt.  You only need spend ten minutes with it to realize how well executed and how well meshed everything in this Distro works.  It is a joy to use.

If you don't need Ubuntu, but would like to stay in the Ubuntu repo and remain lean, yet have a 'Windows-like' experience, then I would suggest you consider giving Linux Mint 15 Xfce a test drive.   

I have spent enough time now with Mint 15 Xfce to say that it approaches perfection.  

-- Dietrich

Related articles

• Xubuntu 13.10 Beta 2 (Saucy Salamander) Screenshot Tour
• Development Release: Ubuntu 13.10 Beta 2

Read More

Wanna Get Your Point Across...? Show 'Em The "Magic"

My Uncle Ernie was a jokester and a magician.

When he and my cousins would make the drive down from Wisconsin, I waited patiently at the kitchen table, hoping he'd show me some of his tricks before we ate dinner.



Uncle Ernie rarely disappointed us.

Simple magic really...misdirection to make stuff disappear, card tricks, quarters found in your ear.  It's all stuff we roll our eyes at as adults, but to a 10 year old kid...

My Uncle Ernie was Merlin Incarnate.

Of course, to the magician, it's an ability...just like driving a car or riding a bicycle.  You are able to do it better, directly proportional to the amount of practice you put into it.

A few years back, I was active on the forums of a particular Linux distro.  I also spend a good deal of time advocating for said distro so of course I wanted it to succeed.

At this particular time, there was a fairly well known bug that caused evolution to open upon an email field, even if Thunderbird was your default email client.  There was a fairly complex set of instructions to correct the problem but a lot of new users were put off by having to hand edit obscure configuration files to fix it.

I don't think it took me more than 15 minutes to write a script to fix the broken system call and implement it.  It was a dirty little hack...code most any self-respecting geek wouldn't let see the light of day, but it did the job and that's all I really cared about.  I messaged the forum poster and told him how to make the script active.  With the click of a mouse, his problem, and this problem in general was relegated to "Problems that used to be".

When this new user asked me how I had fixed it, I hesitated for a moment or two...thinking that any explanation would be wasted on the uninitiated...pearls before swine as it were.

But I did indeed explain to him how a dirty little script hack had corrected the problem.  He was not only amazed...it opened his eyes to the real beauty of Linux and Open Source Software.

He now understood that if this problem had been a Windows issue, he would have had to wait until someone at Microsoft corrected the problem.  It might have been fixed, or it might not have been fixed...

With Linux, all he had to do was mention it on a forum and it was fixed on the spot.

I will let him reveal himself if he wishes...he is well-known in his distro circles and does a lot of good work both in the kernel and in the user space.  Not everyone has the opportunity to show others the magic trick.  Most people don't care...they just want the problem gone.

Like a funny noise in the motor of their car.

Like the squeaky door that doesn't respond to oil or lubricant.

Like the computer that won't open the right program when you click the trigger link.....

But every now and then, you show someone how the magic trick works and it intrigues them.

And that, in turn...

It can change history.

Related articles

• Valve Announces SteamOS Linux Distro, Download Soon and Free Forever

Read More

Is NIST History?

by Dietrich Schmitz

In my last story, Is OpenSSL's Cryptography Broken?, I reported the ongoing developments surrounding a suspect security problem with the implementation of openssl.

The story, unfortunately, continues to unfold with suspicion now turning to confirmation in a NY Times report that the NSA inserted altered random number generator code into the Dual Eliptic Curve Deterministic Random Bit Generator so as to predict private key encoding and provide a 'backdoor' entry point mechanism.  (Image credit: fearlessmen.com)

Despite strong denials coming from the National Institute of Standards and Technology (NIST) who oversaw the development of the Eliptic Curve Cryptography (ECC) standard, many now are left having a strong distrust of the agency.  From a The Register story NIST publicly responded:

The US National Institute of Standards and Technology (NIST) has vehemently denied accusations that it deliberately weakened encryption standards to help the NSA's monitoring activities.

"We want to assure the IT cybersecurity community that the transparent, public process used to rigorously vet our standards is still in place," said NIST in a statement.

"NIST would not deliberately weaken a cryptographic standard. We will continue in our mission to work with the cryptographic community to create the strongest possible encryption standards for the U.S. government and industry at large."

The statement from NIST said that working with the NSA was 'standard operating procedure' and required by law.  In an attempt to throw a 'wet blanket' on the bonfire, NIST has reopened the standard for public comment.

Regardless, one outspoken Developer, Bruce Schnierer, said in a podcast:

NIST took a big credibility hit unfortunately. There are good people there doing good work but we don't know which of their standards are tainted, we don't know how much collaboration there is with the NSA. 

And unfortunately because trust is lost when they get up and say the NSA doesn't affect our standards we don't believe them. We need a way to get back trust.

In other news the IETF offered up a 'fool-proof' plan to PRISM-proof the Internet.

What is the take-away?

Cryptography standards have all now been put into question in addition to the public relations disaster that confronts NIST.

Whether or not NIST will recover remains to be seen as it is quite likely that all cryptography standards will require rigorous audits.

In the meantime, the prevailing perception is that many cryptographic standards have been compromised and privacy is not assured by virtue of their use on the Internet.  As such, it will take a significant amount of time to pragmatically review each standard and thoroughly vet code before a level of confidence in these needed privacy measures will be restored.

And, the question of whether or not trust should be placed in agencies such as NIST is now the main focus and primary concern.  Is NIST history?  Only time will tell.

-- Dietrich

Related articles

• Is OpenSSL's Cryptography Broken?
• NIST says it did not 'deliberately weaken' encryption standards to help NSA undermine Internet security
• How far did the NSA go to weaken cryptography standards?
• NIST's Random Bit Generator Algorithm Fail

Read More

Is OpenSSL's Cryptography Broken?

by Dietrich Schmitz

Last month, in early August, a colleague Friend of mine, +Scott Doty contacted me.  He expressed his concern regarding Red Hat's implementation of OpenSSL.

The issue brought to my attention by Scott concerns a specific bugzilla ticket which was opened in 2007 and has never been addressed.

I offered Scott to reach out to Red Hat's public relations the same day he contacted me.  The answer returned the following day was essentially a 'no comment' and that I should refer to the comments section on the ticket -- deemed to be 'self-explanatory'.  If you take the time to review the ticket, you'll see where Scott appended his own comments in the August time frame toward the bottom.  It's fairly long.

Red Hat had absolutely no intention of fixing the bug, specifically, regarding the treatment of the Elliptic Curve Cryptography implementation in OpenSSL, and according to the comments on the ticket they felt ECC was patent encumbered.

Yet, in other sources on the Internet, one can find reference to a 'work-around' which would avoid any IP infringement issues.  Quoting from Wikipedia.org's ECC page:

"...However, according to RSA Laboratories, "in all of these cases, it is the implementation technique that is patented, not the prime or representation, and there are alternative, compatible implementation techniques that are not covered by the patents."[3] Additionally,Daniel J. Bernstein has stated that he is "not aware of" patents that cover the Curve25519 elliptic curve Diffie–Hellman algorithm or its implementation.[4] RFC 6090, published in February 2011, documents ECC techniques, some of which were published so long ago that even if they were patented any such patents for these previously published techniques would now be expired...."

Alright, so it struck Scott as being odd that such a bug was laying around collecting dust, and I agreed.

In the meantime, we have seen a series of news releases with Snowden giving out new information.  One of the claims has been that the capability of NSA to penetrate presumed to be secure cryptography standards has become much improved to such an extent that they are now collecting information flowing over SSL with impunity and have broken a few other cryptographic standards, purportedly.  I say this only because it's Snowden's word vs. the NSA and the NSA is completely 'mum' on the topic.

The sensational news story "Report: NSA Can Break Internet Encryption"arrived last week and created quite a stir.  The title is a carefully crafted wording.  Naturally, it is quite an unsettling thought to have all presumed Internet security breached, but the story's author hedged a bit at the end of his story saying:

"...Despite the NSA's ability to crack web encryption with these means, Wired's Kim Zetter notes that "these methods don’t involve cracking the algorithms and the math underlying the encryption, but rather rely upon circumventing and otherwise undermining encryption." 

And Snowden himself said during a Q&A with The Guardian in June that cryptography works. 

"Properly implemented strong crypto systems are one of the few things that you can rely on," he said...."

Now, the distinction to be made ties into the title of the story -- namely that, provided that an 'implementation' of strong cryptography coded 'properly' with no side-effect bugs cannot be hacked.

Put another way, bug-laden cryptography can result in weakening of the underlying cipher's strength and so can potentially be cracked.

This would seem to suggest that the NSA have found defects in various cryptographic standards, or, by whatever means, have introduced themselves intentionally crafted bugs in such a way to induce such weakening, thereby achieving their end-goal to crack encryption methodologies.

This led me to think more about OpenSSL and that languishing buzilla ticket.  Just yesterday, I had an exchange with +Jan Wildeboer to whom I regularly communicate, usually on Google Plus.  I broached the matter of the Red Hat OpenSSL bugzilla ECC ticket with him and curiously enough, today, he cc'd me with this Google Plus post (thank you Jan):

Mike Hearn

Shared publicly  -  10:29 AM

A few days ago Bruce Schneier, who has reviewed the leaked Snowden documents, warned against the use of elliptic curve cryptography on the grounds that it requires users to agree on curve parameters and he no longer trusts the parameters to not have back doors. Specifically he's talking about the NIST curves. NIST is a US organisation that was previously widely respected and considered trustworthy.

However, his warning seemed to be based more on general conservatism than any specific intelligence cleaned from the leaked documents. We know the NSA has tried to subvert the standards setting process and we know they may have advanced mathematical attacks that the public doesn't know about. ECC requires various constants to be agreed on globally for an instantiation to be used. Hence, the concern.

But that isn't specific evidence. Unfortunately, today I  learned (via Gregory Maxwell) that the process for selecting the "random" curve parameters appears on the surface to be completely implausible. The parameters are the output of SHA1, which should be good if the seed was selected in a reproducible manner. But they were not. The seeds are extremely large constants with no explanations of where they came from. That smells very strongly of something that might be hacked.

It gets better. It turns out that these constants are not only unexplainable but were actually generated by an employee of the NSA. And it turns out that the IEEE working group that worked on standards for ECC was actually holding its meetings on the NSA campus and its membership therefore had to be approved by the NSA as well.

At this point it is fair to assume that the NIST SECG curves should be abandoned for all uses. Bitcoin uses secp256k1 which was not selected in the same way and is more likely to be OK, and besides the NSA is unlikely to care about stealing peoples wallets (we don't use ECC for secrecy, just authenticity). And luckily academics like djb and Tanja Lange have created new variants of ECC independently of the NSA which are technically better anyway. But the upgrade process away from the SEC curves is going to be a pain.

So, that's quite interesting.  It would now appear that ECC is borked and quite possibly has been so for quite some time, thanks to the handy-work of the NSA.

The cat is now out of the bag.  I am now wondering how many other cryptographic standards need a thorough audit and scrubbing of any questionable code and fixing of languishing bugs?

-- Dietrich

Related articles

• On the NSA
• NSA's (and GCHQ) Decryption Capabilities: Truth and Lies
• Google confirms Java and OpenSSL crypto PRNG on Android are broken.
• Good SSL for your website is absurdly difficult in practice

Read More