Blog
An interview with Citus Data: Scaling out Postgres
Posted By: PostgresOpen on Wednesday, September 13
We just wrapped up a great several days with many from the Postgres community. But, just because the conference is over doesn't mean it all has to come to an end. In coming weeks, we'll have videos from the talks up, and we are already starting our preparations for next year.
But first, we took a chance at the conference to sit down with our other Diamond sponsor Citus Data to learn a bit more about them and get a better sense of why they think Postgres is so important.
PostgresOpen SV: Can you start by giving us a quick overview of what Citus does?
Craig Kerstiens: Citus is an extension to Postgres that makes it easy for you to scale out. We do this by sharding your data seamlessly under the covers and distributing your data across many nodes. Scaling out gives the database more memory, more compute, and more storage — oh, and we can parallelize the SQL, too—and as a result, your application gets a lot more performance. At the same time, to your app, Citus still looks like a standard single node Postgres database. And we've made Citus available as open source, as enterprise software, and as a fully-managed database as a service.
PostgresOpen SV: Citus has been around for over five years now, but it seems you're moving a lot faster now. Has something changed or can you share a bit on that progress?
Craig Kerstiens: It turns out building a distributed database takes time. I'd venture to say that building a robust solid database takes between 10-20 years. By leveraging Postgres, we were able to shave a good bit of that time off, and we've been rapidly innovating on Citus so it works just as you'd expect Postgres to work, only: distributed. The uptick you’ve seen from Citus over the last year is probably the result of a bunch of different things: First, we open sourced Citus, which has made our technology available to so many more people. Second, because we made Citus available as a fully-managed database as a service, we’ve been able to connect with a lot of customers who quite frankly would like to forget about their database, and focus instead on their application. And third, the team. The distributed systems experience we have from engineers that were early Amazon.com and AWS engineers coupled with many of us on the database as a service side of things that helped build and scale Heroku Postgres lets us consolidate that expertise then give it right back to our customers which creates a nice cycle.
PostgresOpen SV: I noticed at the conference you talked a lot about multi-tenant and SaaS apps, what's unique or different about those type of apps?
Craig Kerstiens: When it comes to sharding, there are a lot of different opinions out there on how hard or easy it is. And most of those opinions are right in their own way, the reality is that your data model has a lot to do with what type of sharding is right for you and your application. The good news for the SaaS applications out there is that many of them have a multi-tenant data model, and it turns out the way we scale out Postgres works really well for these multi tenant apps.
PostgresOpen SV: What’s next for Citus?
Craig Kerstiens: More Postgres, more scale, more features. There are a ton of options ahead for us, but the key theme is that we want to let you get back to building your app and not worrying about the database and putting out fires. We’ve consistently heard from customers after migrating that they have not thought/worried about their database after migrating whereas before it was a constant area of pain. We want to keep making that experience easier to come to, and give you the tools to work more flexibly with your data.
An interview with Microsoft: Adding Postgres to Azure and joining the Postgres community
Posted By: PostgresOpen on Wednesday, September 06
As our conference kicks off tomorrow we're looking forward to all the great talks and conversation about all things Postgres. To wet your appetite we took the chance to sit down with another of our Diamond sponsors, Microsoft, who recently added support for Postgres to Azure. Give a read to see how they approach Postgres and the community.
PostgresOpen SV: Microsoft has long been known for SQL server, but recently added PostgreSQL to the ranks for Azure. Can you tell us a little more on the background of how PostgreSQL came to be part of the ranks for Azure?
Rohan Kumar:Microsoft’s mission is to enable every person and every organization on the planet to achieve more. To make this mission meaningful for our customers, we intend to meet them where they are, helping them to be productive with the technologies and tools of their choice. PostgreSQL has a strong community and is one of the most loved open source databases, bringing industry leading innovations to customers. As we move towards offering customers more choice, bringing PostgreSQL as a fully managed service on Azure was a natural step for us.
PostgresOpen SV:Microsoft has made a lot more commitments to open source in recent years. PostgreSQL of course has a long history as an open source database, how do you see Microsoft contributing and engaging with that community in the future?
Rohan Kumar: We are excited to be working with PostgreSQL community. We would love to partner with the community to bring our experience, from building SQL Server over the years, to PostgreSQL – and to learn in areas where PostgreSQL excels. We have already engaged on pgsql-hackers mailing list and working with the community on patches. Moving forward, we will continue to contribute back and partner with the community in the service of our customers. As we look forward, the possibilities of what we can work together on are amazing.
PostgresOpen SV: While PostgreSQL is newer to Microsoft, databases are certainly not new having a long history with SQL Server which is also a very solid database. How do you expect all of your learnings over the years with SQL Server to apply and support PostgreSQL?
Rohan Kumar: As mentioned above, we would love to share our learnings from working on SQL Server with the PostgreSQL community. While there are many areas that we can work on together with the community, a couple of areas to highlight would be connectivity for the cloud and making PostgreSQL more robust and compatible in Windows development environment.
PostgresOpen SV: For those looking to get started with PostgreSQL on Azure, where should they start?
Rohan Kumar: The best way to get started with Azure Database for PostgreSQL is to visit our documentation page where you can find 5-minute quickstarts, step-by-step tutorials and samples. If you are ready to go, you can start using the $200 free credit when you create a free account with Azure.
An interview with 2ndQuadrant: whats new and whats coming in Postgres
Posted By: PostgresOpen on Thursday, August 31
Continuing on down our list of some of the great sponsors we get to sit down and talk to, today we jump in with Simon Riggs of 2ndQuadrant.
PostgresOpen SV: 2ndQuadrant works both with a bunch of US clients as well as contributing to much of Postgres and the ecosystem around it. How does the mix of contributing to core Postgres as well as direct client interaction help improve Postgres overall?
Simon Riggs: 2ndQuadrant is a major developer of open source PostgreSQL, contributing both major new features and bug fixes regularly. We use our direct client interactions to focus these contributions so we can prioritise the biggest issues for our clients. With over 15 years of customer engagements we have been able to identify bottlenecks and then contribute towards enterprise features that fix them in PostgreSQL, benefiting everyone that uses it.
PostgresOpen SV: Postgres 10 is just around the corner, 2ndQuadrant have many developers that contributed to it. Can you share a bit on what you're most excited about in Postgres 10?
Simon Riggs: It’s the Big One - the release where we finally get logical replication into core!
2ndQuadrant has supported BDR and pglogical for some years now, though we now have some very important aspects of that added to core PostgreSQL. The new logical replication uses two new commands: PUBLICATION and SUBSCRIPTION to allow users to define flows of changed data from one server to another.
This work has taken many years of effort to bring to the current level, resulting from many 2ndQuadrant developers and various reviewers and testers. It opens up many possible use-cases with this important new replication technology.
PostgresOpen SV: Because you are an active participant in the community you have a good idea for where the project is headed beyond Postgres 10... What’s the future look like?
Simon Riggs: Very good! Spoiler alert - we win!
Open source PostgreSQL has been growing year on year and we’re attracting interest from the largest enterprises now. There are still some large predators out there but the community is big enough and strong enough to push forwards.
Technically we continue to tune, innovate and evolve. Expect to see many further performance features, new ways of developing and ease of use features - whatever customers need to succeed with PostgreSQL.
From 2ndQuadrant you can expect some big new developments around Bi Directional Replication (BDR), though we also have things brewing around Postgres-XL 10, Barman, repmgr 4, pglogical 3 and pgbouncer as well as other existing projects.
Recent developments include the release of OmniDB, a new web-based admin tool for both PostgreSQL and other DBMS, specially designed around rapid response and ease of use.
We have pursued a joint strategy of providing both working code (available now) and later submitting the features into core PostgreSQL. Postgres-BDR is an open source project from 2ndQuadrant that provides multi-master replication for PostgreSQL. One of the greatest achievements to come out of our work on BDR is the logical replication technology that will be available in PostgreSQL 10, as I previously mentioned. We will be submitting further enhancements of performance and robustness into later releases of PostgreSQL. You can read more about the Postgres-BDR roadmap here.
PostgresOpen SV: What are you doing about cloud?
Simon Riggs: We’ve been encouraging cloud usage for many years now with contributions such as energy saving low-power mode, EXTENSIONs, database block checksums, group commit and a slew of backup and replication features all specifically aimed at cloud usage.
We’re happy that various cloud vendors have been using PostgreSQL in the cloud for many years now and we believe it’s important for users to have both choice and quality.
2ndQuadrant has offered consulting services and support to customers using cloud services for many years. 2ndQuadrant offers the Trusted PostgreSQL Architecture (TPA) tool to orchestrate and deploy database services for on-premise, public or hybrid clouds. 2ndQuadrant uses TPA to offer Managed and Supported DBaaS to customers, particularly around Postgres-BDR and Postgres-XL, as well as standard Postgres.
PostgresOpen SV: Do you see a lot of people moving to PostgreSQL from other databases? If yes, how do you help them with the migration process?
Simon Riggs: PostgreSQL is the most popular database when it comes to new adoption. Even existing users of other databases are flocking towards PostgreSQL - and it’s not just to save money. PostgreSQL is fully capable of securely supporting enterprise database systems.
We have developed specialized tools to facilitate migrations from Oracle and other database management systems. Our team not only consists of the largest collective of PostgreSQL experts globally, they also have a deep knowledge of other DBMS such as Oracle and MySQL.
2ndQuadrant believes in promoting open source PostgreSQL as a solution that is trusted globally by large and small businesses in various industry verticals, highlighting that it's developed by a vibrant community focused on making it more powerful with every release. Using PostgreSQL allows users to break free from vendor lock-in and design a database that is customized for their business needs, something that any company would benefit from.
Announcing Keynote Guest: Paul Vixie to PostgresOpen SV
Posted By: PostgresOpen on Friday, August 25
We are thrilled to announce that Dr. Paul Vixie, CEO of Farsight Security, Internet Hall of Fame Innovator, and one of the primary individuals responsible for the one of the largest, most distributed and used databases in the world, the Domain Name System (DNS), will be our Guest Keynote speaker this year!
Paul helped create one of the most important databases in the world: DNS, a key backbone technology for the ongoing operation of the Internet and he brings an interesting perspective on both data and open source. We're excited to hear from Paul and learn from his insights in addition to our other keynote speakers from: 2ndQuadrant, Citus Data, and Microsoft.
If you're still thinking of joining us there make sure to grab one of the few remaining tickets before they're sold out.
An interview with Heroku on Postgres
Posted By: PostgresOpen on Tuesday, August 22
We got the chance to sit down with Rimas Silkaitis from the Heroku Data team and dig into some of the history behind their choosing and becoming the first database as a service provider and how the see things evolving.
PostgresOpen SV: Heroku Postgres helped drive much of the resurgence of Postgres, why do feel it was often overlooked several years ago?
Rimas: It depends on where it was being overlooked and the time frame. If we're talking about Heroku's sweet spot, building web applications, 10 or more years ago, then Postgres was definitely overlooked. Postgres took the approach of focusing on security, being standards-compliant and having a robust feature set. Other databases at the time, focused on ease of use. For many developers looking to build web applications, and for many for the very first time, ease of use was paramount. Because of the approach the community took on Postgres, it provided fewer opportunities for developers to shoot themselves in the foot as their application grew.
PostgresOpen SV: You've been running a database as a service for some years now, in fact you were one of the first to run Postgres as a service, how have you seen managed database services change?
Rimas: When Heroku Postgres started being able to run Postgres as a service or any database for that matter was no small feat. That was the value of the service. Over time what we've seen is that getting the database service running is the minimum bar. Any DBaaS needs to go beyond just providing the service and do more for the customer. This means providing more features and functionality around operating the database that may not be inherently built into the database itself. For example, Heroku Postgres has commands and functions that will diagnose potential problems with the database called PGDiagnose. It's those kinds of features that developers are really looking for so that they don't have to worry about the database.
PostgresOpen SV: For most people trusting you with their data is a big decision, what are they able from a database as a service that they couldn't achieve otherwise.
Rimas: I'm a bit biased because I lead Heroku Postgres but using a DBaaS should be a no-brainer. So much goes into running a database effectively that if you want to do it on your own you're going to need so much infrastructure to pull it off. That's just the fixed costs. What about the people power needed to keep that database running? But the real value for anyone using a DBaaS is not all of the features and processes that comes from buying that product but that time that you and your team gets by focusing on the areas that matter to the business. Will you make money by building your own infrastructure to support a high-availability setup or does it come from time your team takes to build the application that actually generates revenue? The really good DBaaS providers will make sure that they give you and your company the confidence that your data is secure by providing all of the necessary transparency and data portability. This way you can be confident that you are in control.
PostgresOpen SV: What are you most excited for about the direction of Postgres and its future?
Rimas: We're really excited about the work being done around partitioning. In postgres 10, the community is making it easier to use and making it part of the database itself. While table partitioning does not get Postgres in the distributed system realm, it does point to a future where Postgres could get there. Companies like Citus and projects like TimescaleDB are already showing us that it's possible. Postgres will become the database choice to handle all sorts of applications and their use cases, it's only a matter of time.
PostgresOpen SV Schedule Live
Posted By: PostgresOpen on Tuesday, August 08
PostgresOpen Silicon Valley this year is the exciting combination of PostgresOpen and PGConf Silicon Valley. The conference is coming up right in downtown San Francisco, September 6th to 8th! Our schedule is live and we thought we could take a look at a few of the highlights. Feel free to take a look yourself and get your ticket and room while they're still available.
This year the schedule boasts talks from a number of Postgres contributors such as:
- Magnus Hagander taking a look at Postgres 10
- Bruce Momjian teaching us about Window Functions
- Andres Freund examining Postgres efficiency improvements
- Joe Conway looking at the PostgreSQL STIG
- Marco Slot looking at Postgres as a distributed computing platform
- Robert Treat highlighting whats available with logical replication
- Andrew Kane from Instacart looks at how Postgres could index itself
- Jesse Willett of Prosperworks walks through how they scaled beyond a single node by sharding with Citus
- Stephen Rees from Pandora shares how they run Postgres
Rooms are available only for a few more days so make sure to grab your reservation now and we look forward to seeing you there.
Early Bird Registration Opens!
Posted By: PostgresOpen on Monday, May 15
Early Bird Registration for PostgresOpen SV 2017, being held in beautiful downtown San Francisco from September 6th to 8th, is now open! PostgresOpen and PGConf SV have joined forces this year to put on this fantastic conference in downtown San Francisco.
Simply go to: our tickets page and register to attend the longest running annual PostgreSQL conference in the US.
The Program Committee is excited to be able to offer tickets for PostgresOpen at the same rate as last year, with a $200 discount for early bird registrations!
We also want to remind you that the Call for Papers is only open until May 30th, Anywhere on Earth (AoE), this is the last chance for you to submit your talk for PostgresOpen SV 2017, there's only two weeks left!
Presentations on any topic related to PostgreSQL including, but not limited to, case studies, experiences, tools and utilities, application development, data science, migration stories, existing features, new feature development, benchmarks, and performance tuning are encouraged.
Tutorials will be announced in the coming weeks- watch our blog for updates!
The Program Committee looks forward to bringing the best PostgreSQL presentations and tutorials from speakers around the world the fantastic Parc55 in downtown San Francisco.
Speakers will be notified by June 6th, 2017 AoE, with the schedule to be published once selected speakers have confirmed.
PostgresOpen SV 2017 is only able to happen with the support of our fantastic sponsors. We are extremely pleased to be able to recognize our Diamond launch sponsors: 2ndQuadrant and CitusData and head to our site to see all of our Gold, Silver and Supporter sponsors on our sponsors page.
Sponsorship opportunities are still available! Please visit sponsorship campaign page to review our prospectus.
We look forward to seeing everyone in San Francisco!
Call For Papers
Posted By: PostgresOpen on Friday, March 24
The Call For Papers for PostgresOpen SV 2017 is now open!
PostgresOpen and PGConf SV have joined forces this year to put on a fantastic conference taking place in downtown San Francisco from September 6th to 8th.
If you are working with PostgreSQL, please submit a talk!
Presentations can be on any topic related to PostgreSQL, including, but not limited to, case studies, experiences, tools and utilities, migration stories, existing features, new feature development, benchmarks, performance tuning, and more!
The 2017 PostgresOpen Committee looks forward to bringing the best PostgreSQL presentations and tutorials from speakers around the world to San Francisco!
We're only able to do that with support from our great sponsors!
We are extremely pleased to announce our Diamond launch sponsors for PostgresOpen 2017: 2ndQuadrant and CitusData and our Gold sponsors, VividCortex and Amazon Web Services!
Keep an eye on our sponsors page for updates.
If you are interested in sponsoring, please visit our sponsoring campaign.
PostgresOpen is an Associated Project of the United States PostgreSQL Assocation (PgUS)! Without PgUS, we would not be able to run these conferences each year, nor support the many PUGs around the country, so please visit them and consider becoming a member to help promote the growth and education of PostgreSQL in the US!
Anyone and everyone in the PostgreSQL community is encouraged to submit a talk.
Talks will be accepted up until May 29th, Anywhere on Earth (AoE), also known as: 2017-05-29 23:59:59-12:00.
To see when this is in your timezone, simply use PostgreSQL:
=> set timezone = 'US/Eastern'; -- Change to yours!
SET
=> select '2017-05-29 23:59:59-12:00'::timestamptz;
timestamptz
-------------------
2017-05-30 07:59:59-04
Speakers will be notified by June 5th, 2017 AoE, with the schedule to be published once selected speakers have confirmed.
Early Bird registration for PostgresOpen 2017 will open on May 15th!