Guy Lafleur franTarkenton

Trino's Efficiency in Processing JSON Files in Data Lakes: A Technical Deep Dive

Introduction

Trino, the distributed SQL query engine formerly known as PrestoSQL, is engineered for high-performance, interactive analytics across a multitude of heterogeneous data sources.1 Its architecture is particularly well-suited for querying large datasets residing in data lakes, whether deployed on-premises using HDFS or in the cloud on object storage systems like Amazon S3, Google Cloud Storage, or Azure Blob Storage.2 A key capability enabling this is Trino's schema-on-read approach, allowing users to query data in various formats directly where it resides, without requiring upfront transformation and loading into a proprietary storage system.3

JSON (JavaScript Object Notation) has become ubiquitous in modern data ecosystems, frequently used for API payloads, application logs, configuration files, and semi-structured data exchange.7 However, querying JSON efficiently at scale presents significant challen

Enter this in the search box along with your search terms:

Get all gists from the user santisbon.
user:santisbon

Find all gists with a .yml extension.
extension:yml

Find all gists with HTML files.
language:html

My Openshift Cheatsheet

List all non openshift/kube namespaces

requires jq CLI

oc get namespaces -o json | jq '[.items[] | select((.metadata.name | startswith("openshift") | not) and (.metadata.name | startswith("kube-") | not) and .metadata.name != "default" and (true)) | .metadata.name ]'

Project Quotes, Limits and Templates

RESTful API know-how

Motivation

I place my learning process in this document with 2 motives:

To have a quick guide whenever I lost the track of knowledge.
To share the knowledge with anyone wants to learn RESTful APIs

1. Before, some theory

To remove a submodule you need to:

Delete the relevant section from the .gitmodules file.
Stage the .gitmodules changes git add .gitmodules
Delete the relevant section from .git/config.
Run git rm --cached path_to_submodule (no trailing slash).
Run rm -rf .git/modules/path_to_submodule (no trailing slash).
Commit git commit -m "Removed submodule "
Delete the now untracked submodule files rm -rf path_to_submodule

	from __future__ import with_statement
	from alembic import context
	from sqlalchemy import engine_from_config, pool
	from logging.config import fileConfig

	from models import Base

	config = context.config

	fileConfig(config.config_file_name)

	# Boolean
	op.add_column('projects', sa.Column('is_closed', sa.Boolean(), server_default=sa.schema.DefaultClause("0"), nullable=False))

	# DateTime
	op.add_column('projects_users', sa.Column('created_at', sa.DateTime(), server_default=sa.func.current_timestamp(), nullable=False))