Home » Programming
Category Archives: Programming
Linux Documentary
Cristian Moldovan, Revolution OS Linux Documentary 2001 Full Movie, here.
REVOLUTION OS features interviews with Linus Torvalds, Richard Stallman, Bruce Perens, Eric Raymond, Brian Behlendorf, Michael Tiemann, Larry Augustin, Frank Hecker, and Rob Malda.
Will Python Kill R?
Stephen O’Grady, RedMonk, Will Python Kill R? here.
In an article entitled “Python Displacing R As The Programming Language For Data Science,” MongoDB’s Matt Asay made an argument that has been circulating for some time now. As Python has steadily improved its data science credentials, from Numpy to Pandas, with even R’s dominant ggplot2 charting library having been ported, its viability as a real data science platform improves daily. More than any other language in fact, save perhaps Java, Python is rapidly becoming a lingua franca, with footholds in every technology arena from the desktop to the server.
Cache Sizing
Marcus Wittman, Marcus Wittman’s Blog, Schoenauer Vector Triad, here. Verify Cache size by performance drop off in vectorized fused multiply add. Lead Farmers wired Anderson Varejao since he will have to split burn with Bynum until he (could be Bynum or AV) gets injured and now he doesn’t look like the 14-14 double double stud he was briefly last year. Turkeys have a better chance of making it through Thanksgiving than AV’s burn in CLE. Pulled James Anderson from the wire since he is getting 35+ min burn at the SG in Philly. Patrick Beverley kind of killed it in the loss to Philly. HOU at MSG tonight – Patrick Beverley feasts on the Knick backcourt?
OS X 10.9
John Siracusa, ars technica, OS X 10.9 Mavericks: The Ars Technica Review, here.
Table of Contents
Knight Capital Americas LLC
Alexandra Stevenson, NYT, Knight Capital to pay $12 Million Fine on Trading Violations, here.
On the morning of Aug. 1 2012, Knight Capital, Wall Street’s biggest trading firm, sent out a wave of accidental stock orders – more than four million – that reverberated through the market and eventually resulted in a $460 million loss for the firm.
SEC, In the Matter of Knight Capital Americas LLC Respondent, here. The dead “Power Peg” code activation on a failed production roll distribution, described here, seems real hard to test in UAT. Bet that it’s a good time to talk to Appel about Verified Software Toolchain if you are the GETCO folks who just purchased Knight and are paying the 12 million USD fine. Not obvious to me that the failed production roll distribution process is going to be in scope for VST, but who knows, maybe they thought about that. I suppose you want the gateway, FPGA or otherwise, connecting you to the exchange verified, right? Locking that up could be lucrative and the sales pitch is sort of straight forward, once you read the SEC/Knight Cap. post mortem.
On August 1, 2012, Knight Capital Americas LLC (“Knight”) experienced a significant error in the operation of its automated routing system for equity orders, known as SMARS. While processing 212 small retail orders that Knight had received from its customers
SMARS routed millions of orders into the market over a 45-minute period, and obtained over 4 million executions in 154 stocks for more than 397 million shares. By the time that Knight stopped sending the orders, Knight had assumed a net long position in 80 stocks of approximately $3.5 billion and a net short position in 74 stocks of approximately $3.15 billion. Ultimately, Knight lost over $460 million from these unwanted positions. The subject of these proceedings is Knight’s violation of a Commission rule that requires brokers or dealers to have controls and procedures in place reasonably designed to limit the risks associated with their access to the markets, including the risks associated with automated systems and the possibility of these types of errors.
John Carmack and John Romero + Wireshark on Qt + MS 3Q + Robotics talks
Jeff Atwood, Coding Horror, You Don’t Need Millions of Dollars, here.
Masters of Doom is the story of John Carmack and John Romero creating the seminal games Wolfenstein 3D, Doom, and Quake.
Gerald Combs, Sniff free or die, The official Wireshark blog, We’re switching to Qt., here.
Today I released the next development version of Wireshark, 1.11.0. This marks a major change in the direction of the project. We’re switching our user interface library from GTK+ toQt. Both libraries make it easy for developers write applications that will run on different platforms without having to rewrite a lot of code. GTK+ has had a huge impact on the way Wireshark looks and feels and on its popularity but it doesn’t cover our supported platforms as effectively as it should and the situation is getting worse as time goes on.
Michael J. Moore, Bloomberg, Morgan Stanley Tops Estimates on Stock Trading, Brokerage, here. FI revenue down 43%.
Third-quarter revenue from fixed-income sales and trading, run by Michael Heaney and Rob Rooney with commodity trading co-heads Colin Bryce and Simon Greenshields, was $835 million, excluding DVA. That compared with estimates of $975 million from JPMorgan’s Abouhossein and $1 billion from Credit Suisse’s Chen.
Fixed-income revenue fell 43 percent from $1.46 billion in the year-earlier quarter. This year’s figure compared with $1.29 billion at Goldman Sachs and $2.78 billion at Citigroup Inc.
Evan Ackerman, IEEE Spectrum, Stanford-Berkeley Robotics Symposium 2013, here.
If you want to relive the symposium minute by minute, skip down to the bottom of this post to watch all four sessions in their entirety. For a schedule of who presented what when, click here. We’re going to highlight five of the presentations, focusing on research that we’ve been following in the past, but it’s absolutely worth spending some time listening to what everybody has to present: these are top-notch robotics researchers, and it was (frankly) amazing to get them all into one room at the same time.
RGM Advisors Programming Test
RGM Advisors, Order Book Programming Problem, here. Nice focus.
Problem
Your task is to write a program, Pricer, that analyzes such a log file. Pricer takes one command-line argument: target-size. Pricer then reads a market data log on standard input. As the book is modified, Pricer prints (on standard output) the total expense you would incur if you bought target-size shares (by taking as many asks as necessary, lowest first), and the total income you would receive if you sold target-size shares (by hitting as many bids as necessary, highest first). Each time the income or expense changes, it prints the changed value.
Knuth Volume 4A
Knuth, The Art of Computer Programming, Volume 4 A, here. Calling a top in bitcoin here. If nobody from the “algorithm currency” bitcoin can solve Knuth’s problem then the universal dystonic constant has been exceeded. It’s like not having Rodney Dangerfield involved in a currency based on catch phrases. Also, who steals money from Knuth’s bank? There’s only 275 check routing numbers to search for, you would think there is a simple algorithm to identify the guy with the felonious buddies. Presumably someone out there owns the SQL servers in Knuth’s bank. Is it too much to ask to find the guy who screwed this up for everybody?
Separate item – If you could send someone back in time to change just one thing, it would be send Dennis Ritchie from 1969 back to when Knuth was starting work on MIX in 1968 and convince him to use C instead of MIX, right? Immediate advance of like 15 years in STEM across the board if you just change that single decision.
Financial Fiasco
Leading banks and investment funds have been foundering, because of bad debts and lack of trust; and other, less well-known kinds of fiscal chaos are also on the horizon. For example, due to an unfixable security flaw in the way funds are now transferred electronically, worldwide, it is no longer safe to write personal checks. A criminal who sees the numbers that are printed at the bottom of any check that you write can use that information to withdraw all the money from your account. He or she can do this in various ways, without even knowing your name — for example by creating an ATM card, or by impersonating a bank in some country of the world where safeguards are minimal, or by printing a document that looks like a check. The account number and routing information are all that international financial institutions look at before deciding to transfer funds from one account to another. (See, for example, Grant Bugher’s comments.) More and more criminals are learning about this easy way to acquire money, and devising new schemes to conceal their identities as they steal the assets of more victims.
Nowadays almost everybody knows that it’s dangerous to reveal your credit card number, or to have that full number on a printed document that somebody might find in the trash. Soon people will learn that it is equally dangerous to reveal the numbers that are printed in plain sight on every check. Forget signatures; banks have no time to verify them. The once venerable system of checking accounts is irretrievably broken. Before long, companies will find it impossible to give out paychecks without exposing themselves to unacceptable risk.
One consequence of this debacle is, alas, that I can no longer write checks to reward the people who discover errors in my books. The system that I’ve been using has worked well for almost forty years; but recently I have had to close three checking accounts, and the criminal attacks on those accounts have caused significant grief to my bankers. (Certainly I do not believe that anybody who received one of my checks has been in any way a culprit. But all such recipients are entitled to bragging rights; therefore the numbers printed on those checks inevitably become known to random members of the public.) I cannot in good conscience continue to traumatize the people at my bank, who obviously have plenty of other things to worry about.
After painful deliberation I’ve come up with a new plan, which I hope will be acceptable to all concerned, and perhaps even welcomed as an improvement. Instead of rewarding heroic bug-finders with dollars, I shall henceforth award brownie points, otherwise known as hexadecimal dollars (0x$). From now on it will be kudos, not escudos.
Instead of writing personal checks, I’ll write personal certificates of deposit to each awardee’s account at the Bank of San Serriffe, which is an offshore institution that has branches in Blefuscu and Elbonia on the planet Pincus.
It turns out that only 9 of the first 275 checks that I’ve sent out since the beginning of 2006 have actually been cashed. The others have apparently been cached. So this change in policy will probably not affect too many people. On the other hand, I don’t like to renege on promises, so I shall do my best to find a suitable way to send money to anyone who really prefers legal tender.
Everybody who has received a reward check or a hexadecimal certificate from me since 1 January 2006 automatically has an account at the Bank of San Serriffe, and these accounts are listed on the bank’s website. All of these people have my undying gratitude for the invaluable help they’ve generously provided in order to improve the books and the software that I’ve written. I ask friendly readers to keep sending those precious bug reports, and to let me know if my new policy displeases you in any way.
Algorithms
Dasgupta, Papadimitriou, and Vazirani, Algorithms, 2006, here.
Once you think about Algorithms in this way, it makes sense to start at the historical beginning of it all, where, in addition, the characters are familiar and the contrasts dramatic: numbers, primality, and factoring. This is the subject of Part I of the book, which also includes the RSA cryptosystem, and divide-and-conquer algorithms for integer multiplication, sorting and median finding, as well as the fast Fourier transform. There are three other parts: Part II, the most traditional section of the book, concentrates on data structures and graphs; the contrast here is between the intricate structure of the underlying problems and the short and crisp pieces of pseudocode that solve them. Instructors wishing to teach a more traditional course can simply start with Part II, which is self-contained (following the prologue), and then cover Part I as required. In Parts I and II we introduced certain techniques (such as greedy and divide-and-conquer) which work for special kinds of problems; Part III deals with the sledgehammers of the trade, techniques that are powerful and general: dynamic programming (a novel approach helps clarify this traditional stumbling block for students) and linear programming (a clean and intuitive treatment of the simplex algorithm, duality, and reductions to the basic problem). The final Part IV is about ways of dealing with hard problems: NP-completeness, various heuristics, as well as quantum algorithms, perhaps the most advanced and modern topic. As it happens, we end the story exactly where we started it, with Shor’s quantum algorithm for factoring.
Correctness
Nicole Hemsoth, HPCWire, Lustre Founder Spots Haskell on HPC Horizon, here. The Haskell FPGA Compiler should be interesting.
The saying goes that in Haskell, the function is a first-class citizen–and this status might make it a solid fit for a range of high performance computing environments. Despite what Braam admits is a daunting learning curve, there is an open field of possibilities for Haskell to infiltrate HPC. As it stands, there is an active community around it and around 5000 open source and tools available. But the real value for high performance computing, he argues, lies in Haskell’s productivity and correctness–a worth that’s been validated in select industrial use cases.
Arguably, Google and Facebook have brought more attention to Haskell in recent years, but there are a number of other notable uses that highlight Braam’s confidence in the functional language. For instance, Chicago-based Allston Trading, a high frequency trading company, uses Haskell in their trading infrastructure. AT&T is using it in their Network Security group to automate internet abuse complaint processing. Bank of American is using it in their backend data transformation and loading system and Credit Suisse’s Global Modeling and Analytics Group has been using it since 2006 to improve modeler productivity and open access to those models across the organization.
Peter Braam, CUFP, The Awesome Haskell FPGA Compiler, here.
The use of hybrid CPU/FPGA systems shows great promise for dramatically improving the performance of data-intensive applications. In addition, a new generation of application switches with integrated FPGAs provide the ability to run high-performance and mission critical applications in the network itself. We believe the major impediment to full scale deployment of these systems is the time and effort required to design and implement integrated hybrid solutions.
Parallel Scientific’s Awesome Haskell FPGA Compiler provides the environment that allows a developer to express hardware solutions in a high-level Domain Specific Language (DSL). The environment allows software simulation and testing of the solution in an interactive environment, thus avoiding the extremely long development cycle times traditionally required for FPGA design. Once the design has been verified in software, the compiler and tool chain create industry standard Verilog which then feeds into the FPGA vendor’s existing tool chain.
Dustin Sklavos, Anand Tech, Haswell and GK110 vs. Ivy and GK104: DigitalStorm Virtual System Review, here.
If you’re like me, you were probably incredibly underwhelmed by initial reviews of Haswell. Ivy Bridge proved to be a decent overclocker, but Intel’s miserly switch from fluxless solder to thermal paste as a thermal interface material in their chip packaging put a hard limit on what we could really do with it, and they’re continuing that aggavating trend with Haswell. One of the most frustrating results is a flattening of the overclocked performance curve from Sandy Bridge to Ivy Bridge, and thankfully we can at least test and see today if Haswell does anything to change things.
Ian Cutress, Anand Tech, Intel Z87 Motherboard Review with Haswell: Gigabyte, MSI, ASRock and ASUS, here.
Pink Iguana 