I relate to this so much. I have experienced the same things, and I also find myself moving across to data engineering.
In my case, I was in organisations that wanted data science, but had no capability or interest in supporting the role, so a lot of my time and effort has been having to put down the data science tools, and learn devops, software development and data engineering so that I can get back to the point where I do my data science work.
I’ve also become frustrated with my data science peers lack of knowledge about the surrounding fields- I get that being a top tier software dev isn’t the primary responsibility of a DS, but it would certainly make their life, and the life of everyone around them a lot easier if they did make an effort. There’s a sense of “learned helplessness” in parts of data science (and parts of data engineering too) in which “if some third-party tool can’t do it for us, we just can’t do it” and imagination is limited to the features the latest framework de jour offers.
In my case, I was in organisations that wanted data science, but had no capability or interest in supporting the role, so a lot of my time and effort has been having to put down the data science tools, and learn devops, software development and data engineering so that I can get back to the point where I do my data science work.
I’ve also become frustrated with my data science peers lack of knowledge about the surrounding fields- I get that being a top tier software dev isn’t the primary responsibility of a DS, but it would certainly make their life, and the life of everyone around them a lot easier if they did make an effort. There’s a sense of “learned helplessness” in parts of data science (and parts of data engineering too) in which “if some third-party tool can’t do it for us, we just can’t do it” and imagination is limited to the features the latest framework de jour offers.