Cloud Infrastructure, Generative AI, AWS, Azure, GCPIf cloud platforms were people (thoughts from a Generative AI)Can you relate to these? As of writing this blog post, I have been really enjoying the new releases in Google’s Gemini AI for both images…
Cloud Infrastructure, Data Engineering, Python, DuckDBQuackpipe, running DuckDB local and on AWS GlueQuackpipe Stop waiting for AWS Glue jobs while in development by using a local-first workflow with DuckDB and deploy with Terraform. Your…
Cloud Infrastructure, Data Engineering, Python, SalesforceQuery Salesforce with PythonQuery SalesForce objects with Python Ever found yourself exporting data from Salesforce using the Web UI over and over again. Ever wished…
Python, Data Science, Software DevelopmentPublishing a Python Streamlit app to Posit ConnectPython applications published to Posit Connect This is a simple Streamlit application structured as a modular project and managed using…
Cloud Infrastructure, Data Engineering, GCP, Machine Learning, Python, Software EngineeringMaking Machine Learning predictions from BigQuery DataformNFL Touchdown Prediction with Dataform and BigQuery ML This blog post demonstrates how to build an end-to-end ML pipeline using Google Cloud…
Cloud Infrastructure, Data Engineering, GCP, Machine Learning, Python, Software EngineeringThe Tale of Two Routes: A Love Story Between BigQuery and Vertex AI for Machine Learning predictionsThis blog post is the first, and probably last where I thought I would see what the Github Copilot VSCode extension could do with writing a…
Cloud Infrastructure, MLOps, Machine Learning, PythonGCP Serverless prediction service using Cloud Run from a Vertex AI modelDeploying ML Models on Cloud Run: From Placeholder to Production This guide walks through deploying a machine learning model on Google Cloud…
Cloud Infrastructure, Data Engineering, DevOpsChecking the public IP address of a GCP Cloud FunctionPublic IP address of a GCP Cloud Function When you deploy a Cloud Function without configuring a VPC, it operates within Google’s internal…
Generative AI, Software EngineeringIf programming languages were people (thoughts from a Generative AI)Can you relate to these? Inspired from these (https://leftoversalad.com/c/015_programmingpeople/), let’s see what a Generative AI thinks of…
Cloud Infrastructure, Data EngineeringDebugging Dataform with Node.js in VScodeDebugging Dataform with Node.js in VScode Debugging Dataform JavaScript If you happen to be a data engineer who is not familiar with the…
Cloud Infrastructure, Data EngineeringGCP Cloud Functions for Web Scraping AFL dataScraping web data using GCP Cloud Functions The main reason for this blog post is a practical example of writing a GCP Cloud Function with…
Software Development, Data Science, PythonFlask application with custom package deployed to GCP Artifact Registry and Cloud RunFlask application with custom package deployed to GCP Artifact Registry and Cloud Run What we are looking to do in this example is…
R, Data Science, Web DevelopmentA way to build CRUD apps with R Shiny and SQLiteBackground Often as software engineers working within organisations we are coerced into using a software stack that we may not have chosen…
Teradata, Data Engineering, GCP, BigQueryCopying a table from Teradata to Google BigQueryCopying a table from Teradata to Google BigQuery For this demo I will be using a Windows PC including: Teradata Studio Teradata ClearScape…
Teradata, Data EngineeringConnecting Nifi to TeradataConnecting Nifi to Teradata For this demo I will be using a Windows PC including: Docker desktop PowerShell Core 7.3.6 VMWare Workstation…
Analysis, EDA, Graph, PythonHip-hop collaboration network graphIntroduction As a Hip-hop fan (and other genres of music) it is obvious that certain artists are more alike than others. I was interested to…
Web Development, Cloud Infrastructure, System AdministrationFlask application with Postgres containerized across multiple cloudsFlask app with a Postgres database containerized across multiple clouds Background We are building out an application that has a decoupled…
Data Engineering, PythonConnecting to Teradata from PythonReading and Writing data between Python and Teradata This blog post is paired with the associated Github repository https://github.com…
Software Development, Data Science, PythonPython mono repo for Data Science with PoetryMono repo with Python Poetry What we are looking to do is create a code repository that can be used as the source for many Python…
System Administration, Data Science, RInstall Posit Connect on UbuntuOverview We want to test out the installation and management of Posit Connect (previously RStudio Connect). Prerequisites A machine running…
System Administration, RPublish an R Package to a private repository and useOverview Let’s say we want to manage an R code base as a package. The benefits might include: a good way to distribute R code for other…
Cloud Infrastructure, Data Science, System AdministrationDatabricks security model architectureOverview Let’s say we have a brand new Databricks resource that we have just created from Azure. It comes as an empty shell ready for use…
Cloud Infrastructure, Data Science, System AdministrationDatabricks and Azure DevOps CICD Pipeline for DBFSDevOps CICD Pipeline for Databricks This is in reference to this Git repo https://dev.azure.com/mortimer-xyz/mortie23/_git/dbfs-cicd-pieline…
Data ScienceDatabricks using Azure ML model registryRegistering a model in AzureML from Databricks What if we want to use AzureML as a central model repository for all models across many…
Cloud Infrastructure, Data ScienceModel scoring API as a Docker containerUse Docker to host a Python FastAPI service In my preview blog post (Model scoring API) I went through the technical basics of a model…
Data ScienceDatabricks sample NFL MLFlowFit a decision tree model for predicting win or loss The code this post is relating to is here: https://github.com/mortie23/databricks-nfl …
Data Engineering, Data ScienceDatabricks sample NFL datasetCreate a Databricks database with sample NFL data The code this post is relating to is here: https://github.com/mortie23/databricks-nfl-data…
Data Science, Web DevelopmentModel scoring API examplesA basic example of a predictive model scoring API The associate Github repo for this blog post is https://github.com/mortie23/fastapi-linear…
Business Intelligence, Software DevelopmentPower BI PBIX Git version controlHow to commit a Power BI PBIX to Git This is in reference to the associated https://github.com/mortie23/pbix-git-version-control What is the…
CI/CD, Web Development, System AdministrationGit post-receive hook for deployment of a static siteUsing a post receive hook for website deployment Other deployment methods Firstly, let’s mention that there are other potentially easier…
Software DevelopmentRaspberry Pi time lapse serverBuilding a time lapse server with a Raspberry Pi and a webcam I had a Raspberry Pi lying around that I use to use as a media server, and an…
Business IntelligenceDynamic Images in Power BI DashboardsDynamic images in Power BI Getting images to be dynamic, such as changing in reaction to an change in the selected value of a slicer seems…
Data Science, Web DevelopmentIntegrating Azure Cognitive with RetoolWhat is Retool Retool is a simple tool for creating a web application. One of the things that makes Retool great is you’re never limited by…
System Administration, Cloud Infrastructure, Data EngineeringInstalling Prefect Server on AzureReading Firstly I came across the following docs on how to install prefect server Prefect Server 101 Deploying to GCP Deploying to a Single…
Software DevelopmentSecret Santa with Node.js and GmailWhat is Secret Santa an arrangement by which a group of friends or colleagues exchange Christmas presents anonymously, each member of the…
Software Development, PythonHow I develop Python in VSCodeWhy Working on Javascript code in Visual Studio Code (VSCode) for the best part of last year got me use to all the good stuff…
Software DevelopmentGit CleanupWhat is the issue? Collaborating on a repo and someone accedientally commits a very large file that is not very compressable. It makes the…
Cloud Infrastructure, Data Science, System Administration, RRStudio with SSL proxyWhat are we doing We have a server hosting RStudio, and we want encrypt the traffic too and from the RStudio service. We also do not want to…
Business Intelligence, Web Development, Data EngineeringTableau Web Data Connector Node GeneratorUsing Tableau Web Data Connector (WDC) as the data source for dashboard. This post is in reference to the associated Github repo at https…
Data ScienceAgent Based ModellingI was experimenting with Agent based models and came across a really great talk Jackie Kazil | Agent based modelling in Python. This lead me…
Business Intelligence, GeospatialGeospatial shapes for vizualisationPutting analysis on a map People always seem to like seeing data visualised on a map. Something about looking at a gradient of colour on a…
Cloud Infrastructure, Web DevelopmentSetting up a cloud webserver for hosting a static siteWhy would I want to do this? Since there are many easier (options) for hosting a static website (of which I recommend and also use Github…
Business Intelligence, Data EngineeringTableau Web Data ConnectorThis post is in reference to the associated Github repo at https://github.com/mortie23/tableau-wdc. Tableau WDC Quick example of a Tableau…
Business Intelligence, Web DevelopmentEmbedded BIWhat is Embedded BI When people talk about Embedded Business Intelligence (BI) they are usually referring to putting things such as…
Web DevelopmentSpeed Up My SiteOriginally I built my personal site by using a free responsive template and then I just modified the source code. The page loads were slow…
System AdministrationLinux Users GroupsThis post is in reference to the associated Github repo at https://github.com/mortie23/linux-users-groups. Deploy a security model (users…
