micaleel

Data Quality Monitor (DQM) — Phased Execution Plan

Scope: V1 implementation
Basis: dqm_technical_design.md
Runtime: PySpark
Primary catalog/source environment: Hive Metastore-compatible tables
Configuration: YAML

---

micaleel

Data Quality Monitor (DQM) — Technical Design

Status: Draft
Version: 0.3
Scope: V1
Runtime: PySpark
Primary catalog/source environment: Hive Metastore-compatible tables
Configuration: YAML

---

micaleel

Review this PR as a skeptical senior engineer. Do not modify any files.

First understand the intended behavior from the PR diff and surrounding code.

Investigate:
- correctness and possible logic errors
- assumptions and invariants
- affected call sites and interfaces
- edge cases and failure modes
- backward compatibility

micaleel

NCF

Autoencoders for retrievals

Unsupervised-Image-Retrieval/Unsupervised Image Retrieval.ipynb at master · nathanhubens/Unsupervised-Image-Retrieval · GitHub

TODO

• https://github.dev/tensorflow/recommenders/tree/main/tensorflow_recommenders/metrics

Overview

There are difference between GMF, MLP AND NeuMF

micaleel

Bookmarked

Reviewed

• https://github.com/KinWaiCheuk/Triplet-net-keras
• https://github.com/MaurizioFD/RecSys2019_DeepLearning_Evaluation

Pending

• https://github.com/PreferredAI/cornac

micaleel

Explore how embeddings are created with Autoencoders
* https://blog.eto.ai/vector-similarity-search-with-duckdb-44dec043532a
* https://medium.com/kirey-group/autoembedder-training-embedding-layers-on-unsupervised-tasks-fc364c0f6eec
* https://www.kaggle.com/code/marlesson/autoencoder-embedding-for-food/notebook

micaleel

regs = regs or [0, 0]
user_input = Input(shape=(1,), dtype="int32", name="user_input")
item_input = Input(shape=(1,), dtype="int32", name="item_input")

user_embeds = Embedding(
    input_dim=num_users,
    output_dim=embed_size,
    embeddings_initializer="normal",
    embeddings_regularizer=l2(regs[0]),
    input_length=1,

micaleel

% solvers for DEs in ode.m

% TODO Insert relevant values into `range` and `initial_conditions`
% NOTE initial_conditions is a vector (ordered collection) of input values that will
%      unpacked in ode.m file into the X, L, S, N, and H variables respectively.
range = [1:50]
initial_conditions = [600, 3, 0, 3, 3]

# Use ode45 solver
[time_solution, var_solution] = ode45(@ode_sys, range, initial_conditions)

micaleel

% General constants obtained from Section 3.1
mu_X_max    = 0, 275
K_XS        = 0.0624
K_iXS       = 12.40
K_XN        = 0.0812
K_iXN       = 0.6250
K_I         = 45.50
K_iI        = 297.75
A_0X        = 0.6175
E_aX        = 25.9243

micaleel

from typing import List, Tuple
import numpy as np
import os
import pandas as pd


DATA_DIR = ...

data = pd.read_csv(
    os.path.join(DATA_DIR, "ml-100k", "u.data"),