The legal basis is there to protect wildlife from man-made disruption and provides a kind of ecological basis to limit the kind of boundless growth that politicians appeal to.
Unfortunately, for those laws to be effective, they have to be strong enough to beat the various legal shenanigans / loopholes which can be used by developers to effortlessly leapfrog them.
Finally, if the laws are strong enough, they might be effortlessly wielded to prevent even reasonable developments from occurring.
The law lands somewhere in the middle and I think there are always people at either extreme trying to take advantage.
Blocking high-speed rail in the name of conservation makes me want to bash my head against a wall. Guess we'll buy 10000 diesel trucks to move goods north-south instead. Car-world isn't good for conservation, rail is. It's missing the forest for the trees. These laws are selectively invoked by interest groups, they don't event serve the legitimate cause of conservationists - there is enough ammo in UK law to block any development.
* Experience required in both data engineering and back-end engineering spaces
* Experience welcome in ML engineering and data science spaces
I've worked on this team for a year or so now and it's fair to say that there's been no shortage of things to work on - you'll be immersed in the challenge of computing and distributing personalised recommendations across the business via operational and analytical workloads.
You will also become a part of a company which adopts a lot of the nice danish working practices (family first).
The office culture is an important aspect of the business so 3 days a week is a needs-must.
Why go this route, rather than incorporating GenAI provided by OpenAI, Alphabet, Athropic, etc.?
It seems to me that we are at a transition point, similar to the transition of self-hosting to cloud-hosting, where bespoke recommendation systems will no longer be a competitive edge. Or am I betting wrong?
As the position is for a lead engineer, it wouldn't be unusual for applicants to have ideas or a view on more recent developments in this technology area and/or it's application to this particular problem space...that's all I'll say
* Experience required in both data engineering and software engineering
* Experience welcome in ML engineering and data science
I've worked on this team for a year or so now and it's fair to say that there's been no shortage of things to work on - you'll be immersed in the challenge of distributing personalised recommendations across the business and become a part of a company which adopts a lot of the nice danish working practices. The office culture is an important aspect of the business so 3 days a week is a needs-must. If this sounds like your bag - apply below:
bandcamp is a bit more of a level playing field - the artists can decide whether they want to use the platform to stream or sell their music, while being given some control over the number of streams per listener before a song will be unavailable to stream (although this does lean on cookies, I think, so not difficult to get around...)
Longtime data engineer - web apps in vanilla HTML/CSS/JS with Node - and some fun with ncurses in C++. Copilot is helping me out a bit but I’m having fun!
Hey, you're like my evil twin! Longtime web dev here who always wanted to learn more data engineering.
HTML/CSS/JS is fun and creative to some degree, but kinda abstract. Data engineers get to work with real-world phenomena! What prompted the interest in web stuff?
I never did proper "data engineering" per se, but we did have casual run-ins with geodatasets and GIS, and used them to build a home solar calculator that takes a ZIP code and estimates that area's available yearly sunlight, utility rates, electricity consumption, etc. and combine it into an estimated size for a home solar system. It was like a primitive version of Google Sunroof (https://sunroof.withgoogle.com/), which is much, much better.
I too love the intersection of data and visualization/usability, turning obscure spreadsheets into useful public interfaces :) The geospatial world is just one part of that. It used to be cool, but I guess nowadays it'd probably be more about turning data into LLM-digestible training sets so that they can more directly analyze questions and answer them in plain language.
Do you think traditional DE is still worth learning, with Skynet on the horizon? What's a good way to get started?
Wow - that sunroof thing is cool, I bet that was a fun area to work on.
There’s a lot to be said for data engineering when done at scale - systems design, devops, cloud engineering etc.
I’m not sure that’s going yet…
Building the individual units (reliable ETL pipelines) might be on the chopping block in the near future - I’m not convinced there’s 100 competing ways to build these.
Still - best way to get started is probably to get stuck in with running a data pipeline using a locally deployed Airflow instance. Read some data from some api and write it to a local database deployment (postgres?).
May I ask the same of web application development?
> that sunroof thing is cool, I bet that was a fun area to work on
Yeah, it was (and is) really cool! To be clear, I didn't work on that, just something similar but much more primitive and with much simpler datasets. Google did a really good job there.
> locally deployed Airflow instance
I'm pretty comfortable with basic ETL stuff, but never used Airflow. Will have to look into that, thanks!
> May I ask the same of web application development?
I can give you my opinion, but that's all it is. I'm not a FAANGer and primarily work with small businesses and nonprofits, so my perspective may be biased and incomplete. I should also note that I'm also kinda an AI-optimist, meaning I have a much more positive view of both its competence and its threat than many people. But ultimately, I'm a nobody, just a rando on the internet, so take all this with a big grain of salt :)
In the time I've been doing web stuff (20-30 years, depending), I've seen the industry move towards higher and higher levels of abstraction. In the old days (the 90s and early 2000s) it was a lot of hacked-together HTML + backend logic, then it gradually moved towards the clientside (frontend) with things like ActiveX, Flash, and eventually CSS and JS.
JS eventually won out and you can build really amazing apps almost entirely in the frontend now (like Figma, Photopea, OpenSolar, Felt, and other frontend-heavy, UI-driven things like those).
But those are what I'd consider proper "apps". There are also may websites that are just, well, sites and not what I'd consider apps, things like your average news or blog site, or maybe basic ecommerce stuff. That's where I see the most abstractions/consolidations into a few big frameworks, like Shopify or Wordpress + WooCommerce, or Wix/SquareSpace/Weebly for simpler sites, or headless CMSes + Jamstacks for more complex sites (disclaimer: I currently work for a headless CMS company). I think these sorts of sites are the most at risk of automation, which really has been happening for decades already. What used to take weeks of setup and then tons of ongoing maintenance (both on the frontend and backend) is basically just a one-click deploy these days with little need for actual coding anymore. And even complex, bespoke UIs are increasingly being AI-driven... the company that makes Next.js (a big frontend framework) is also using AI to try to replace their own customers (us frontend devs), lol: https://v0.dev/
I think there will always be a few humans needed in the loop, but probably fewer and fewer over time, and usually at the "architect" level, where you design overall systems and write some minor glue code to tie it altogether, but don't need to dive too deeply into the nitty-gritty anymore. I'm especially scared for junior coders, because even today, Copilot and ChatGPT are already way better than most of them (and often, better than myself too). One moderately experienced coder with a few AIs can easily replace what used to take a team, and do so with much less overhead (no need for Agile crap, three layers of mid-management, everlasting meetings, etc.). I think it's going to get a lot leaner, which means fewer openings, more productivity per remaining dev, but fewer dev openings overall.
But that's only if we assume that current web devs stay the course and don't upskill/adopt more and more AI practices. Just like fewer and fewer of us work in PHP or ASP or Ruby on Rails these days, I would assume that many would sidestep and just incorporate more AI and GPT into their workflows. Maybe there's some opportunities there for lean + mean startups who can do a lot more with fewer employees than before. But still, AI/ML is a fundamentally different enough skill set (like actual CS + math + modeling stuff, not just gluing together UI code) that a lot of us are going to flunk out and join the breadlines.
Even at this early stage of AI, I'm fairly confident that my career as a small-biz frontend dev is a dead end, with maybe 4-5 years left if I'm really lucky. It's not just because of AI doing a better job, necessarily, but the simple hype around AI means a lot of the money that used to be in Web is now pivoting towards AI. The bubble's burst, and sure, there'll always be a few dev jobs here and there, just like there are still newspaper or graphic design jobs here and there, but I'm fairly certain its heyday is over. Just my 2¢ =/
That said, I don't think it ever hurts to learn HTML + CSS at least. Those are relatively simple, declarative markup languages that's really more similar to Markdown than programming. JS (and especially React, etc.) is where it gets tricky, but even that ecosystem is finally somewhat mature/stable, such that it's a pretty easy time to get started, having missed the craziness of the late 2010s and early 2020s where it was going through very rapid iterations. Today it seems to have stabilized around React + Next.js as the go-to framework (by popularity), and the documentation and examples have gotten better. I wouldn't quit your day job to go learn any of this stuff -- like I don't think it's a good time to dive headfirst into becoming a web dev, with it becoming more and more yesterday's game instead of the future -- but as a side project? Sure. At the very least, it enables better UI design and visualizations, and that's always fun! Even with AI as the companion, there's a lot more creativity (and human psychology) there than gluing together pipelines and APIs (of course, that's probably just my bias as a frontend person creeping through).
Alternatively, it's also possible to get really good at some particular niche in the stack (like Canvas graphics or WebAssembly or WebGL) and find a super-specialist position at a bigger company. That would probably be a safer bet than my generalist background.
I dunno... sorry, I don't mean to be a downer, it's just that tech has always been fast-moving and cutthroat, and it's probably going to become even more ruthless in the future. Once AI starts self-iterating, it'll become exponential and I don't think humans will be able to keep up. Right now it seems like the biggest barrier is simply our hardware manufacturing capacity, but that's going to ramp the hell up soon. But anyway... we can only play the cards we're dealt, as we're dealt them. No need to fret about things out of my control, and Skynet is very much out of my control, lol.
As an old man (nearly 40), I have limited time and neuroplasticity left. It's harder for me to just pivot to the new shiny every few years. But if you're younger, I don't think it's ever wrong to explore and try new things and see what sticks (and what pays)!
Sorry, that's just the bigger-picture overview/rant. If you have any questions about more specific technologies/stacks, I'd be happy to share thoughts on those too.
Thanks for all the web dev advice and tech callouts. Some real neat examples there which I knew nothing about.
Regarding niches, I think as a data engineer, becoming fluent in an interactive front-end data-viz library might be the slam dunk that I'm probably looking for.
Having looked at D3, I am quite impressed! Do you have any experience or knowledge of this lib?
> A dumb phone - small and minimal functionality, easy to forget about
This requires more diligence, but can't you create the experience you want on a smart phone (no social media, disable all notifications, etc.) without giving up utilities that provide actual value, such as GPS / navigation?
> Exercise - no headphones, a gym provides background music and enough human contact to keep boredom at bay
Interesting. I feel like I am more distracted and unfocused when I listen to the gym's music (sometimes their music just doesn't match the intensity of the workout) or when I socialize between workouts.
GPS can be in not so malicious environment, I'm talking about separate GPS device instead of addictionful phone with no buttons, so-called smart but maybe not for end user's brain and free time.
Usually a black screen with a bunch of mostly white letters :)
Seriously though, I do this a lot too. All my instant messaging comes though a text interface, most of my note taking and other work. I use the browser a lot still because it's sadly unavoidable.
If I really need to focus I even use my old VT520 CRT terminal. The soft amber glow really helps to focus <3
If possible, try to spend more time with people who espouse these desired characteristics and less time with those who don't.
Finally, conversation should be...fun...if possible. If you hack your way to being 100% in control of things all the time, then how much fun is that going to be for others?
>" Start with writing."
Many people are very good at writing. When you read them, we get excited to talk to them face to face and discover them till we found out they are poor at verbal communication.
Unfortunately, for those laws to be effective, they have to be strong enough to beat the various legal shenanigans / loopholes which can be used by developers to effortlessly leapfrog them.
Finally, if the laws are strong enough, they might be effortlessly wielded to prevent even reasonable developments from occurring.
The law lands somewhere in the middle and I think there are always people at either extreme trying to take advantage.