Pink Iguana
Advertisements

Logical Induction

Scott Garrabrant, et.al., Machine Intelligence Institute, Logical Induction, here.

We present a computable algorithm that assigns probabilities to every logical
statement in a given formal language, and refines those probabilities over time.
For instance, if the language is Peano arithmetic, it assigns probabilities to
all arithmetical statements, including claims about the twin prime conjecture,
the outputs of long-running computations, and its own probabilities. We show
that our algorithm, an instance of what we call a logical inductor, satisfies a
number of intuitive desiderata, including: (1) it learns to predict patterns
of truth and falsehood in logical statements, often long before having the
resources to evaluate the statements, so long as the patterns can be written
down in polynomial time; (2) it learns to use appropriate statistical summaries
to predict sequences of statements whose truth values appear pseudorandom;
and (3) it learns to have accurate beliefs about its own current beliefs, in a
manner that avoids the standard paradoxes of self-reference. For example, if
a given computer program only ever produces outputs in a certain range, a
logical inductor learns this fact in a timely manner; and if late digits in the
decimal expansion of π are difficult to predict, then a logical inductor learns
to assign ≈ 10% probability to “the nth digit of π is a 7” for large n. Logical
inductors also learn to trust their future beliefs more than their current beliefs,
and their beliefs are coherent in the limit (whenever φ → ψ, P∞(φ) ≤ P∞(ψ),
and so on); and logical inductors strictly dominate the universal semimeasure
in the limit.
These properties and many others all follow from a single logical induction
criterion, which is motivated by a series of stock trading analogies. Roughly
speaking, each logical sentence φ is associated with a stock that is worth $1
per share if φ is true and nothing otherwise, and we interpret the belief-state
of a logically uncertain reasoner as a set of market prices, where Pn(φ) = 50%
means that on day n, shares of φ may be bought or sold from the reasoner
for 50¢. The logical induction criterion says (very roughly) that there should
not be any polynomial-time computable trading strategy with finite risk tolerance
that earns unbounded profits in that market over time. This criterion
bears strong resemblance to the “no Dutch book” criteria that support both
expected utility theory (von Neumann and Morgenstern 1944) and Bayesian
probability theory (Ramsey 1931; de Finetti 1937)

 

Advertisements

Iceland HPC, Intel Thread problem, Intel Lineup

Doug Black, HPC Wire, HPC-as-a-Service Finds a Toehold in Iceland , here.  Cheap Bitcoin farming cycles.

While high-demand workloads (e.g., bitcoin mining) can overheat data center cooling capabilities, at least one data center infrastructure provider has announced an HPC-as-a-service offering that features 100 percent free and zero-carbon cooling.

Verne Global, a company seemingly intent on converting Iceland into a gigantic, renewably powered data center facility, has announce hpcDIRECT,  a scalable, bare metal service designed to support power-intensive high performance computing applications. Finding initial markets in the financial services, manufacturing (particularly automotive) and scientific research verticals, hpcDIRECT is powered by the island country’s abundant supply of hydroelectric, geothermal and, to a lesser degree, wind energy that the company says delivers 60 percent savings on power costs.

Joel Hruska, Extreme Tech, Major Hyper-Threading Flaw Destabilizes Intel Kaby Lake, Skylake CPUs, here.

Now, a major Hyper-Threading flaw has been discovered that can destabilize Intel CPUs based on both Kaby Lake and Skylake — not something Intel needed on the heels of AMD’s new CPUs. The issue is reported to cause “unpredictable system behavior,” which could mean anything from corrupting data to outright system crashes. The issue was picked up by Hot Hardware, via Debian.org. While Debian is a Linux distro, the warning makes it clear that the problem can happen to any operating system and is not limited to Linux.

Intel’s 2018 Roadmap Shows New High-End Cascade Lake-X Debuting Next Year, here.

2017 has been a banner year for CPU launches. AMD’s Ryzen debut in March kicked off its own aggressive hardware ramp, with the Ryzen 5 and Ryzen 3 families following in the spring and summer, and Raven Ridge debuting in the last few weeks. Intel’s Kaby Lake-X and Skylake-X launched in June, with Skylake-X offering faster performance and higher core counts for the same price than Intel had previously shipped. AMD’s Threadripper debuted in August with 16 cores at the same price as a 10-core Intel CPU (and a significant performance advantage), and Intel’s 18-core Skylake-X Core i9-7980XE retook the performance crown (though not the price/performance ratio) in September. Finally, the Core i7-8700K launched in October and won our top-end CPU recommendation, though the Ryzen 7 1800X is still quite competitive in well-threaded workstation workloads.

 

Code

Scope: C/C++ optimizing compiler, ICC, MKL, XLC, MASS, Java, tools, floating point standards, implementation, and benchmarking, Algorithm benchmarking, Anticipate server side is coded in C, data collection coded in Java, Front end coded in  Java + WordPress.

People:  Georg Hager,  Appel,  Kahan,  Hans Mittelsmann,  Agner FogDemmelMuller,

BooksNumerical RecipesHandbook of Floating-Point ArithmeticANSI C StandardK&R, …

PapersWhat Every Computer Scientist Should Know About Floating-Point ArithmeticIEEE 754,, x86 Software developer manual intel.  …

ArchitectureMicrosoft x86Intel Architecture Development manualJava Virtual Machine Specification, …

Multicore ProgrammingIntel Guide. …

CompilersIntel CompilersAMD Developer Central , C++godbolt  (compiler explorer), gcc, …

Optimizers:  AMPL (bwk optimizer framework), GurobiExcelgoogle simplex

Random Number GeneratorsHellekalek Software. …

JavaOracle v8, …

Database/Data Sources: Oracle, mySqlBigtable, …

LibrariesNIST Library of Math Functions,  IBM MASS Lib,  Intel MKL Rquantlibnetlib.org,  nag …

Front EndWordPressTablePressHTML5, Chrome, Safari, FireFox, …

ToolsGithub,  Python,  SciPy.org  tutorialspoint,  google style guide  hacker newsstackexchangestackoverflowXcodeprotobuffer ,…

Operating SystemsLWN.net,  Linux JournalOSX DailyMac OS Wiki, Windows, …

Bibliography v 1.2:

These will typically be the main floating point references used in financial code optimization.

Appel, A. W. (1997). Modern Compiler Implementation in C. Cambridge University Press.

Bundel, D. (2001). CS 279 Annotated Course Bibliography. Retrieved from IEEE 754 summary web page: https://cims.nyu.edu/~dbindel/class/cs279/dsb-bib.pdf

Fourer, R., Gay, D. M., & Kernighan, B. W. (2003). AMPL A Modeling Language for Mathematical Programming (Second ed.). Duxbury Thompson.

Goldberg, D. (1991). What Every Computer Scientist Should Know About Floating-Point Arithmetic. Computing Surveys, Association for Computing Machinery, Inc.

Higham, N. J. (1996). Accuracy and Stability of Numerical Algorithms. Philadelphia, PA: SIAM.

Intel. (2016, June). Intel® 64 and IA-32 Architectures Optimization Reference Manual. Retrieved from https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-optimization-manual.pdf

Intel. (2017, June). Intel® 64 and IA-32 Architectures Software Developer’s Manual. Retrieved from https://software.intel.com/en-us/articles/intel-sdm

Intel. (2017). MKL Development Reference C Bibliography. Retrieved from Intel Developer Zone: https://software.intel.com/en-us/mkl-developer-reference-c-bibliography

Kahan, W. (1997). IEEE Standard 754 for Binary Floating-Point Arithmetic. Retrieved from Kahan’s UCB homepage: https://people.eecs.berkeley.edu/~wkahan/ieee754status/IEEE754.PDF

Markstein, P. (2000). IA-64 and Elementary Functions: Speed and Precision. Upper Saddle River, NJ: Prentice Hall.

Muller, J.-M. (1997). Elementary Functions: Algorithms and Implementation. Boston: Birkhauser.

Muller, J.-M. (2009). Handbook of Floating-Point Arithmetic. Birkhauser.

Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (2007). Numerical Recipies. Cambridge University Press.

This is close to the organization I am looking for.

 

PBC R&D

Pink Iguana will remain primarily a floating point curation site, I will target the links in the blog to the topics listed in the tabs:

  • App PBC – R&D links for the specific Princeton Bank Consortium Applications
  • BigData – Databases, DB hardware, and Data sources for PBC Apps
  • Code – Architecture/Code/Programming links for Compilers and Optimizers
  • EcoFin – Economics and Finance ex quantitative modeling
  • FinQuant – Security cashflow, valuation, and risk models w explanatories
  • FinTech – PBC applications in Fin Tech
  • Infra – PBC Application infrastructure – hardware and software
  • Markets – Primary and Secondary markets and market structure, inc ALM products.
  • Math – Math, algorithms, and CS Theory references and links.
  • Misc. – Project Ideas
  • News – General purpose news links ex Tech, Math and Finance.
  • Tech – General Tech links and research outlets.

Each tab will have links and a bibliography when appropriate. We will use the Blog as a log of our reading and distill the links and information found to be a lasting value in the tabs. The Princeton Bank Consortium will be the production front end for the PBC Applications.

Modeling Cashflows and The Mechanics of Securitization

Keith A. Allman, Modeling Structured Finance Cashflows with Microsoft Excel, here.

Moorad Choudhry and Suleman BaigThe Mechanics of Securitization: A Practical Guide to Structuring and Closing Asset-Backed Security Transactions, here.

Quasi MC Austrians, CashFlow and Waterfall Models, and Duffie

Quasi-Monte Carlo Methods: Theory and Applications, here. Looks interesting.

By ”quasi-Monte Carlo (QMC) methods” we understand all methods in which most carefully chosen quasi-random-point sets are used to carry out simulations in the framework of sophisticated and highly developed modeling environments, to obtain quantitative information in different branches of applications.

The further study and development of QMC methods therefore requires

  • the generation, investigation, and analysis of distribution properties of finite or infinite sequences in all kinds of regions
  • the development, investigation, and analysis of suitable theoretical models on which the applications of the QMC methods are based, and in particular the derivation of error bounds for QMC methods in these models
  • the efficient implementation of the theoretical models and of the algorithms for the generation of the (sometimes very large and high-dimensional) quasi-random point sets, and the provision of  sophisticated software
  • the concrete application of the QMC methods in different areas, the discussion of the implications and of the performance of the applied QMC methods

Consequently, many different branches of mathematics are involved in the comprehensive investigation and development of QMC methods, most notably number theory, discrete mathematics, combinatorics, harmonic analysis, functional analysis, stochastics, complexity theory, theory of algorithms, and numerical analysis. Furthermore, profound knowledge of the branches of applications in which the QMC methods are intended to be used is necessary.

The theory and application of QMC methods is a modern and extremely lively branch of mathematics.  This is demonstrated by an enormous output of research papers on this topic in the last decades, and by the great and growing success of the series of the international conferences on Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing” (MCQMC), which started in 1994 in Las Vegas and was most recently held in Sydney in 2012.

The Austrian research groups initiating this SFB play leading roles in the development of QMC methods. It is the aim of this SFB to intensify the cooperation both between these research groups and with their international partners, to promote new directions and new developments within the theory of QMC methods and their applications, and to train a new generation of highly talented young researchers to carry out research work in the field of QMC methods.

Principia, Structured finance perspectives: Trends in ABS, MBS & CDO Cash flow and Waterfall Models, here.  q2 2013 survey.

Nevertheless, the survey highlighted that despite this dominance, over 65% of investors were using at least one additional cash ow provider. These providers were typically vendors who dedicate themselves to a particular niche asset class, such as Trepp for US CMBS (the one class where Intex was not the leading choice for US investors), or as a competitive alternative, like ABSNet Lewtan and Moody’s Analytics.

This highlights how a diverse portfolio leads to the requirement for a multitude of models which, in turn, can result in several different integration points and stresses to the integrity of system processes.

 

The vendors identi ed were:

  • ABSNet Lewtan
  • Bloomberg
  • Deloitte ABS Suite
  • Interactive Data BondEdge
  • Intex
  • Moody’s Analytics
  • Thompson Reuters
  • Trepp
  • Yield Book

Douglas Clement, Fed Minneapolis, Interview with Darrell Duffie, here.

The Billy Madison Problem

David Silver et.al., arXiv, Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm, here.  I’m going to go ahead and call first, here.

The game of chess is the most widely-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. In contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.

OK, statically load AlphaZero with all the theorems with known proofs in, say, Number Theory. Call this data set INIT.  AlphaZero goes off and teaches itself Number Theory to a super human level. Ask AlphaZero to assign a rating level for the initial static set of all known Number Theory proofs INIT versus its current superhuman level. That level assessment, call it X, is the super human assessment of the documented level of human Number Theory expertise, as of Dec 2017 on earth. Now, find a single new paper to add to INIT that lowers the AlphaZero assessment of human Number Theory expertise from X to  Y < X. That newly added paper is special, it is the Billy Madison paper.

Principal: Mr. Madison, what you’ve just said is one of the most insanely idiotic things I have ever heard. At no point in your rambling, incoherent response were you even close to anything that could be considered a rational thought. Everyone in this room is now dumber for having listened to it.