CFOtech New Zealand logo
Technology news for Kiwi CFOs and financial decision-makers
Story image

Data analytics has changed. Is the financial sector ready?

By Contributor
Tue 23 Jun 2020
FYI, this story is more than a year old

Article by Hitachi Vantara A/NZ vice president and managing director Adrian Johnson.

Economists warn of a deep, long-lasting economic downturn as Australia copes with the COVID-19 pandemic. Financial institutions are going to need every edge they can get. It’s yet another reason why these data-rich companies must turn their vast amounts of unstructured data into valuable assets. Of course, they also have to comply with open banking, ensure data privacy and protect against an onslaught of cyberattacks. 

If that wasn’t enough, here’s one more challenge: the very architecture of data analytics is changing.

Organisations with data lakes and Hadoop environments in place want to get the most out of existing investments, but they do need to consider storage and data management alternatives that will keep pace with changing data analytics requirements, tasks and challenges.

Data lakes giving way to dataops

Data scientists and analysts are now working with and within organisations to identify specific questions to answer, specific business objectives to accomplish and the data sets required to get the job done. It’s certainly a methodology that helps to get a handle on the massive amounts of data generated and collected by businesses today, and the increasing number of data sources they’ve got to consider. It’s a way to find value in the data and contribute directly to business objectives. 

It also means that now there is a lot of work being done on smaller files, or on streaming data. Mining a data lake filled with unstructured data is, therefore, not the best route to effective use of your data. 

Hitachi Vantara A/NZ CTO and presales director Chris Drieberg explains, “A modern dataops approach means that, while data still needs to be managed, categorised, enriched with metadata and governed, and then made accessible to the people who need it, you leave data where it is captured or created rather than spending time, money and effort pulling it into one centralised repository. This begins to make the case for handling processing and storage as separate operations with separate solutions.”

Separation of data processing & storage

Hadoop is a data lake environment that is running into challenges related to scale for many data-rich organisations. It just isn’t an economical storage solution for the sheer volumes of data involved – such as reams of daily data from ATMs, financial transactions, customer surveys and more. More and more nodes can be added, with the associated cost to purchase and manage them, but there is a tipping point after which it takes more effort and cost to maintain this environment, than the value you’re getting out of it. 

As a processing engine and a significant existing investment, Hadoop isn’t going away. But the evolution of data analytics and the storage limitations of Hadoop necessitate pulling the storage engine out of the Hadoop environment in favour of object storage. 

Object storage needs a performance boost

For years now, object storage has been understood to be a scalable and cost-effective solution for archival purposes and to house vast amounts of data to be accessed rarely, or where latency isn’t a concern.

As unstructured data threatens to overwhelm businesses and increasing storage costs have them looking for more economical solutions, more people are seeing object storage as part of the solution to evolving analytics needs.

Says Drieberg, “The beauty of object storage is the metadata. For every file you write into object storage it saves the file plus the metadata about the file. An object storage platform knows the context, file type and all the other machine-generated identifiers plus custom metadata, such as customer name, policy type etc, about each piece of data.”

So object storage stores data in a way that makes it ready and ripe for machine learning, AI and analytics. However, object storage on its own is not a high-performance processing environment.

Hadoop + object storage + VSP

“The super high-performance physical storage capacity, underneath the object storage, will come from the latest in powerful NVMe virtual storage platforms (VSP),” says Drieberg. “VSP uses block storage, which means it knows you’ve written blocks of data to it, and can guarantee you access, policy-based data protection and high performance for your millions of transactions, but it doesn’t know what the data is."

The object storage platform, which sits in front of the VSP, is the piece with the smarts about the data stored within it. Because of the way it stores data, it is a high capacity storage option to service big data analytics. From an infrastructure management point of view, the right VSP will manage everything and give a boost to your legacy systems as well as the object storage.

For the data scientists, analysts and engineers, object storage provides the contextual access to data that lives on the back-end, high-performance block storage, which is the VSP. 

Integrate that with the Hadoop environment as the analytics processing engine and the financial institution’s infrastructure is now ready for next-generation data analytics.

Related stories
Top stories
Story image
Employment
Tech job moves - Forcepoint, Malwarebytes, SolarWinds & VMware
We round up all job appointments from May 13-20, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Vectra AI
Understanding the weight on security leader’s shoulders, and how to shift it
Millions of dollars of government funding and internal budgets are being funnelled into cybersecurity to build resilience against sophisticated threats, indicating how serious this issue has become.
Story image
Cybersecurity
More than 40% of banks worried about cloud security - report
Publicis Sapient's new report finds security and the lack of cloud skills and internal understanding of business benefits are big obstacles for banks moving to the cloud.
Story image
SaaS
Maintaining secure systems with expectations of flexible work
Most office workers feel they've proved they can work successfully from home, and as much as employers try, things aren't going back to the way they were anytime soon.
Story image
Remote Working
How organisations can meet employees' changing expectations
The global employment market has shifted dramatically in favour of employees, sparking the so-called great resignation, in which people are leaving unsatisfying roles in search of greener pastures.
Story image
Digital Transformation
Trading up: It's time to swap core systems for flexible digital applications
This year will see more oranisations planning and commencing high tech renovations that will shake up the way they operate.
Story image
Microsoft
PwC NZ unveils new Cloud Security Operations Center
PwC New Zealand has unveiled its new Cloud Security Operations Center for the entire Microsoft technology stack.
Story image
Talend
Talend introduces new data health solutions for businesses
Talend has announced its latest version of Talend Data Fabric, with the release of Talend Trust Score enabling data teams to establish a foundation for data health.
Story image
Customer experience
Research unveils precarious customer loyalty for retailers
New research has found customers are reassessing established brand loyalties as their priorities and behaviours shift.
Story image
Tech job moves
Tech job moves - Datacom, Micro Focus, SnapLogic and VMware
We round up all job appointments from May 6-12, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Artificial Intelligence
Clear Dynamics closes $35M funding round, invests in global growth
The funding is a major milestone and speaks to Clear Dynamics’ vision for AI-enabled ‘composable’ enterprise software, the company states.
Story image
Vodafone
Vodafone NZ buys remaining stake in retail joint venture
Vodafone New Zealand has purchased the remaining 50% stake in the specialist joint venture (JV) with private equity company Millennium Corp.
Story image
Cybersecurity
NCSC advisory highlights poor security configurations
The GCSB's National Cyber Security Centre (NCSC) has released a cyber security advisory identifying commonly exploited controls and practices.
Story image
Cybersecurity
Hard numbers: Why ambiguity in cybersecurity no longer adds up
As cybersecurity costs and risks continue to escalate, CEOs continue to struggle with what their investment in cyber protection buys. Getting rid of ambiguity becomes necessary.
Story image
Cryptocurrency
Prominent cryptocurrency trader hit by 'perfect storm'
A leading local crypto currency trading platform, BitPrime, says a "perfect storm" has hit its finances, forcing it to put a halt on operations.
Story image
Public Cloud
Cloud adoption still a work in progress, NetApp finds
NetApp has announced the results of the annual Cloud Infrastructure Report based on a survey of public cloud business and IT decision makers.
Booster
Booster Innovation Fund. A fund of Kiwi ingenuity – for Kiwi investors.
Link image
Story image
Remote Working
IT teams deploy powerful technologies to enable remote work
"We found that IT teams mastered the challenges of remote work last year in large part by employing powerful yet easy-to-use technologies."
Story image
Kodari Securities (KOSEC)
NFT trends and opportunities: expert reveals all
The NFT market is growing at an exponential rate, with unprecedented liquidity. Here we explore how businesses can profit.
Story image
Data Protection
Barracuda launches new capabilities for API Protection
"Every business needs this type of critical protection against API vulnerabilities and automated bot attacks," Barracuda says.
Story image
Infrastructure
Nutanix study reveals financial services sector lagging with multicloud adoption
Nutanix has released new research that reveals the financial services sector is lagging behind when it comes to multicloud adoption.
Story image
Artificial Intelligence
CFOs using digital workers and AI to prevent unnecessary loss
New technology is now allowing CFOs to use digital workers to automate their accounting processes, making it easier for them to avoid unnecessary losses.
Story image
Sift
Sift shares crucial advice for preventing serious ATO breaches
Are you or your business struggling with Account Takeover Fraud (ATO)? One of the latest ebooks from Sift can provide readers with the tools and expertise to help launch them into the new era of account security.
Story image
Microsoft
Microsoft unveils adaptive accessories for disability access
Microsoft is introducing an expansive Inclusive Tech Lab to give people with disabilities greater access to technology through new software features and adaptive accessories.
Story image
Artificial Intelligence
SAS launches human-focused responsible innovation initiative
SAS has launched a responsible innovation initiative, furthering its commitment to equity and putting people first.
Story image
SAS
New SAS service overcomes subscription fatigue for media companies
SAS has launched SAS 360 Match which helps media companies move towards a AVOD model to generate revenue as subscribers cancel.
Story image
Digital Transformation
Unlocking the next digital frontier for educational institutions
Understanding where to invest in technology can be challenging for education institutions, especially after the COVID-19 disruptions.
Story image
IT budget
$20m boost for digital technologies announced
The government is spending an extra $20m over four years on its plan to transform the digital technologies industry.
Story image
Sisense
Data and analytics could be key to higher selling prices in APAC
Sisense's latest report has found that almost half of data professionals in APAC think customised data and analytics can create better selling prices for their products.
Story image
Commerce Commission
ComCom welcomes new marketing codes for the telecom industry
The Commerce Commission is welcoming the creation of new marketing codes for the telecommunications industry.
Story image
Artificial Intelligence
SAS unveils AI experience to improve kids' batting abilities
SAS has created The Batting Lab, an interactive experience using AI, computer vision and IoT analytics to help kids improve their baseball and softball swings.
Story image
Excel
Could your Excel practices be harming your business?
While Excel has been the de-facto standard for budgeting, planning, and forecasting, is it alone, enough to support organisations in the global marketplace that’s facing rapid changes due to digital transformation?
Story image
Artificial Intelligence
SAS announces new products amid cloud portfolio success
Analytics and AI company SAS is deepening its broad industry portfolio with offerings that support life sciences, energy, and martech.
Story image
Digital Transformation
The Huawei APAC conference kicks off with digital transformation
More than 1500 people from across APAC have gathered for the Huawei APAC Digital Innovation Congress to explore the future of digital innovation.
Story image
Phishing
Google reveals new safety and security measures for users
Google's new measures include automatic two step verification, virtual cards and making it easier to remove contact information on Google Search results.
Story image
Artificial Intelligence
Updates from Google Workspace set to ease hybrid working troubles
Google Workspace has announced a variety of new features which will utilise Google AI capabilities to help make hybrid working situations more efficient and effective.
Story image
Attain
Revenue operations is taking centre stage
As the business world continues to evolve, new demands need to be met to keep up with the ever-changing landscape. 
Story image
Ransomware
A third of companies paying ransom don’t recover data - report
Veeam's report finds 76% of businesses who are victims of cyberattacks paid the ransom to recover data, but a third were still unable to get their information back.
Story image
Safety
Voxel hits total funding of $18M following ongoing wins
Since raising its seed round in September, Voxel has grown at pace, by decreasing on-site injuries by upwards of 80% and increasing operational productivity.
Story image
Microsoft
Microsoft backing Māori and Pacific wāhine in tech industry
A new initiative focused on getting Māori and Pacific wāhine into the tech industry and backed by Microsoft, NZTech and the government is calling for tech companies to get involved.
Story image
Digital Transformation
Why enterprise records management should be part of any digital transformation strategy
Modern organisations create and rely upon an enormous volume of content, and digital records make up a significant proportion of that content.
Story image
Microsoft
SAS Viya on Microsoft Azure to deliver 204% return - study
The Forrester Total Economic Impact study finds SAS Viya on Microsoft Azure brings a 204% return on investment over three years.
Exabeam
Find out how a behavioural analytics-driven approach can transform security operations with the new Exabeam commissioned Forrester study.
Link image
Story image
Commerce Commission
ComCom appeals $2.25 million fine in Vodafone FibreX case
The Commerce Commission has filed an appeal in the High Court against a $2.25 million fine imposed on Vodafone NZ for its offending under the Fair Trading Act.