Computer Science > Databases
A newer version of this paper has been withdrawn by Hung Ngo
[Submitted on 5 Feb 2013 (this version), latest version 28 Mar 2014 (v5)]
Title:Towards Instance Optimal Join Algorithms for Data in Indexes
View PDFAbstract:Efficient join processing has been a core algorithmic challenge in relational databases for the better part of four decades. Recently Ngo, Porat, R{\textrm{é}}, and Rudra (PODS 2012) established join algorithms that have optimal running time for worst-case inputs. Worst-case measures can be misleading for some (or even the vast majority of) inputs. Instead, one would hope for instance optimality, e.g., an algorithm which is within some factor on every instance. In this work, we describe instance optimal join algorithms for acyclic queries (within $\polylog$ factors) when the data are stored as binary search trees. This result sheds new light on the complexity of the well-studied problem of evaluating acyclic join queries.
We also devise a novel join algorithm over higher dimensional index structures (dyadic trees) that may be exponentially more efficient than any join algorithm that uses only binary search trees. Further, we describe a pair of lower bound results that establish the following (1) Assuming the well-known $\mathsf{3SUM}$ conjecture, our new index gives optimal runtime for certain class of queries. (2) Using a novel, unconditional lower bound, i.e., that does not use unproven assumptions like $\mathsf{P}\neq\mathsf{NP}$, we show that no algorithm can use dyadic trees to perform bow-tie joins better than $\poly\log$ factors.
Submission history
From: Hung Ngo [view email][v1] Tue, 5 Feb 2013 01:24:12 UTC (323 KB)
[v2] Mon, 9 Sep 2013 18:16:36 UTC (1 KB) (withdrawn)
[v3] Mon, 14 Oct 2013 03:35:42 UTC (933 KB)
[v4] Tue, 3 Dec 2013 14:38:38 UTC (1,006 KB)
[v5] Fri, 28 Mar 2014 20:01:57 UTC (1,004 KB)
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.