When you start to build your business it’s often hard to work out what you need in the first 3-6months. A good start is depicted in the diagram. I think in terms of what makes teams productive so Tech Stack is divided between Ops, Product & tech and Sales & Marketing. If you operate in a highly regulated market you could have for example regulatory reporting in the compliance vertical. I constantly aim to review our technology stack to continuously remove legacy.
Category: code
After building online mortgages website and backend systems (integrated with top UK banks) using microservices (Azure, NodeJS, Mongo, React) for the new project I picked a less ambitius stack while leveraging teams know how (Azure, NodeJS, MySQL, templated HTML). This article captures my thinking:
- Fail fast – they let development teams focus on delivering features (to prove or disprove a hypothesis) rather than a complicated microservice architecture
- It helps you to understand your requirements (UML diagrams and domain models are not perfect first time they need to evolve)
- Microservices are complicated to develop (e.g. graceful degradation, health checks, retries) and monitor
- Microservices dependencies are difficult to track
Since going back from Mongo/NoSQL to MySQL, I keep rediscovering the execution order of SQL query.
Inside Microsoft® SQL Server™ 2005 T-SQL Querying
(8) SELECT (9) DISTINCT (11) TOP <top_specification> <select_list>
(1) FROM <left_table>
(3) <join_type> JOIN <right_table>
(2) ON <join_condition>
(4) WHERE <where_condition>
(5) GROUP BY <group_by_list>
(6) WITH {CUBE | ROLLUP}
(7) HAVING <having_condition>
(10) ORDER BY <order_by_list>
The first noticeable aspect of SQL that is different than other programming languages is the order in which the code is processed. In most programming languages, the code is processed in the order in which it is written. In SQL, the first clause that is processed is the FROM clause, while the SELECT clause, which appears first, is processed almost last.
Each step generates a virtual table that is used as the input to the following step. These virtual tables are not available to the caller (client application or outer query). Only the table generated by the final step is returned to the caller. If a certain clause is not specified in a query, the corresponding step is simply skipped.
Brief Description of Logical Query Processing Phases
Don’t worry too much if the description of the steps doesn’t seem to make much sense for now. These are provided as a reference. Sections that come after the scenario example will cover the steps in much more detail.
- FROM: A Cartesian product (cross join) is performed between the first two tables in the FROM clause, and as a result, virtual table VT1 is generated.
- ON: The ON filter is applied to VT1. Only rows for which the
<join_condition>
is TRUE are inserted to VT2. - OUTER (join): If an OUTER JOIN is specified (as opposed to a CROSS JOIN or an INNER JOIN), rows from the preserved table or tables for which a match was not found are added to the rows from VT2 as outer rows, generating VT3. If more than two tables appear in the FROM clause, steps 1 through 3 are applied repeatedly between the result of the last join and the next table in the FROM clause until all tables are processed.
- WHERE: The WHERE filter is applied to VT3. Only rows for which the
<where_condition>
is TRUE are inserted to VT4. - GROUP BY: The rows from VT4 are arranged in groups based on the column list specified in the GROUP BY clause. VT5 is generated.
- CUBE | ROLLUP: Supergroups (groups of groups) are added to the rows from VT5, generating VT6.
- HAVING: The HAVING filter is applied to VT6. Only groups for which the
<having_condition>
is TRUE are inserted to VT7. - SELECT: The SELECT list is processed, generating VT8.
- DISTINCT: Duplicate rows are removed from VT8. VT9 is generated.
- ORDER BY: The rows from VT9 are sorted according to the column list specified in the ORDER BY clause. A cursor is generated (VC10).
- TOP: The specified number or percentage of rows is selected from the beginning of VC10. Table VT11 is generated and returned to the caller.
Therefore, (INNER JOIN) ON will filter the data (the data count of VT will be reduced here itself) before applying WHERE clause. The subsequent join conditions will be executed with filtered data which improves performance. After that only the WHERE condition will apply filter conditions.
(Applying conditional statements in ON / WHERE will not make much difference in few cases. This depends how many tables you have joined and number of rows available in each join tables)
Hard to disagree with this collection of valuable insights. Keep coming back to it https://github.com/i0natan/nodebestpractices
Dare we say we are happy using Blueprint, a ReactJS components library, from Pantir after only 2 months of build.
When manging remote teams I find useful to agree for engineers to perform regular code submissions. I should also note, from my experience on both ends, that when someone — myself included — isn’t making regular pushes or pull requests., they’ve usually “checked out.” That is, they’re not actually working. And more importantly, they’ve stopped caring.
So, as long as you’re actually doing work, and as long as writing code is a daily responsibility of yours, you shouldn’t be afraid to push code every day. If you find that difficult, you might be in the wrong job! And, that’s something both your manager and you should want to see the early signs of!
I created a couple of simple scripts in Node.js: Desk.com / Salesforce API call and store data in Azure SQL for further analysis with Power BI and credit bureau reports.
I default to VS Code IDE that I find clean and efficient. Code is simpler to use compared to full VS as it reduces clutter by making you add what you need rather than switching off / ignoring features that you don’t need.
NPM packages need to be used with caution as majority provide dead ends. My learning is to start solving the problem using baseline language features and only add packages when absolutely necessary.
Prevention is worth a pound of cure.
Most software engineers would vouch for the research showing that problems found early – during design or coding – can be fixed many times faster than when they’re found later in production. We found that initial discussions about how to solve a problem are very helpful to prevent issues downstream.
As Sir Arthur Helps, a 19th century aphorist, said: “Almost all human affairs are tedious. Everything is too long. Visits, dinners, concerts, plays, speeches, pleadings, essays, sermons, are too long.”
My equivalent list for today’s managers would be: “Everthing is too long. Business plans, board presentations, management accounts, sales reports, project updates, standup checkins, system’s architecture, code.”
A lot can be said about designing for cloud-first. Async as default, for example. Chatty DB issues of systems designed with an implicit view that data is instantly available (on-prem DB, for example). However cloud DB suffer from latency causing issues.
A good starting point is to to recognise that different physics is at play in the cloud.