RETS: May 2007 Archives
There's so much more to RETS than can be described here and there are many possible applications for it that many don't even realize yet.
RETS is backed by the National Association of REALTORS® and is here to stay. Work is continuing on RETS 2.0 which has many enhancements over the 1.x versions of RETS.
For more information about RETS and how to get started, visit www.rets.org.
Other links for relevant information mentioned here:
- StandardNames
- RETS-compliant clients
- RETS-compliant server vendors
If there are other questions you have that you can't find answers for, email me and I'll try and point you in the right direction.
RETS is backed by the National Association of REALTORS® and is here to stay. Work is continuing on RETS 2.0 which has many enhancements over the 1.x versions of RETS.
For more information about RETS and how to get started, visit www.rets.org.
Other links for relevant information mentioned here:
- StandardNames
- RETS-compliant clients
- RETS-compliant server vendors
If there are other questions you have that you can't find answers for, email me and I'll try and point you in the right direction.
RETS has a lot of benefits but let's talk about the big ones and the problems they solve.
How do I know how to connect to get the data I need?
With RETS, all you need to know from the MLS is the RETS login URL, your username and your password. From there, you can learn everything else on your own almost entirely using code if you want.
It seems like using a complex system would be harder for an MLS to support. How is this easier?
In most cases, if you have a question about how to log into a typical FTP server and process their data, you'll end up talking to someone that didn't actually create it so your answers will be scattered (if you get an answer at all). Maybe someone went through the MLS software and enabled new fields to be included in the feed. You can't expect everyone at the MLS to know about this so you may be disappointed with the answers you get.
With RETS, it is a more complex system, but it's well documented and as long as both the client and the server are compliant with the standard, you should have little trouble getting setup. You don't need to request help or documentation from them because it's either online (at the rets.org website) or available through the RETS server you're connecting to.
Logging in is pretty easy but then what do I do?
The process is very similar to what you'd do when you first get FTP access to an IDX feed for the first time. Once you login, you figure out what information is available. In RETS, these are called Resources. Once you request the Resources, you'll get back a list of information types that you can learn more about. From there, you can learn about the format of that data and learn how to search it for what you need.
Don't I need to understand the query language used by the MLS's database software to be able to get information?
Nope, that's a major benefit of RETS. RETS uses it's own query language and, when combined with StandardNames, allows you very easy access to pull property information that you need. The RETS server translates your RETS query into something it can understand, pulls the information out of it's database (the way it knows how) and morphs it into the RETS format so it's easy for you to understand when it sends it to you.
If RETS is so powerful and is truly live access to the data, can't I just have a website that connects directly to the RETS server instead of downloading a copy of the data locally?
Sure, and there are products that exist that do just that. Your website might have search criteria for listing price and number of bedrooms. That information could be put into a RETS query, sent to the RETS server, and the response could be displayed back to the user of the site.
The problem is that your MLS could catch onto what you're doing and could limit the amount of information you can pull (using queries) from the RETS server. They may even revoke your access to their system. In most cases, when you want to use data on your website, it's best to download a local copy of the data and access it from there.
You keep mentioning building a website using RETS access. Aren't there other applications for this system?
Absolutely. Having multiple MLS's in the same geographical area provides the potential for those MLS's to share property information between them so that agents in those areas have more to show potential buyers and provides agents with more exposure for their sellers. This is just 1 example. There are many others.
I really, really don't like the MLS software the MLS has for me to access it. Does RETS do anything in that situation?
In short, yes. If your MLS has a RETS server that you have access to, you could use any RETS-compliant program (called RETS clients) to access that data. This introduces more options for agents and brokers because they aren't tied to single user interface.
There are drawbacks, however. In most cases, the MLS software offers a lot of features. So far, it's difficult for a RETS client to offer the same features that the MLS might be able to offer through their own software because their software is made specifically to work with their data in their area. Work is continuing with RETS to try and introduce more capabilities for information sharing.
(continued in Part 6)
How do I know how to connect to get the data I need?
With RETS, all you need to know from the MLS is the RETS login URL, your username and your password. From there, you can learn everything else on your own almost entirely using code if you want.
It seems like using a complex system would be harder for an MLS to support. How is this easier?
In most cases, if you have a question about how to log into a typical FTP server and process their data, you'll end up talking to someone that didn't actually create it so your answers will be scattered (if you get an answer at all). Maybe someone went through the MLS software and enabled new fields to be included in the feed. You can't expect everyone at the MLS to know about this so you may be disappointed with the answers you get.
With RETS, it is a more complex system, but it's well documented and as long as both the client and the server are compliant with the standard, you should have little trouble getting setup. You don't need to request help or documentation from them because it's either online (at the rets.org website) or available through the RETS server you're connecting to.
Logging in is pretty easy but then what do I do?
The process is very similar to what you'd do when you first get FTP access to an IDX feed for the first time. Once you login, you figure out what information is available. In RETS, these are called Resources. Once you request the Resources, you'll get back a list of information types that you can learn more about. From there, you can learn about the format of that data and learn how to search it for what you need.
Don't I need to understand the query language used by the MLS's database software to be able to get information?
Nope, that's a major benefit of RETS. RETS uses it's own query language and, when combined with StandardNames, allows you very easy access to pull property information that you need. The RETS server translates your RETS query into something it can understand, pulls the information out of it's database (the way it knows how) and morphs it into the RETS format so it's easy for you to understand when it sends it to you.
If RETS is so powerful and is truly live access to the data, can't I just have a website that connects directly to the RETS server instead of downloading a copy of the data locally?
Sure, and there are products that exist that do just that. Your website might have search criteria for listing price and number of bedrooms. That information could be put into a RETS query, sent to the RETS server, and the response could be displayed back to the user of the site.
The problem is that your MLS could catch onto what you're doing and could limit the amount of information you can pull (using queries) from the RETS server. They may even revoke your access to their system. In most cases, when you want to use data on your website, it's best to download a local copy of the data and access it from there.
You keep mentioning building a website using RETS access. Aren't there other applications for this system?
Absolutely. Having multiple MLS's in the same geographical area provides the potential for those MLS's to share property information between them so that agents in those areas have more to show potential buyers and provides agents with more exposure for their sellers. This is just 1 example. There are many others.
I really, really don't like the MLS software the MLS has for me to access it. Does RETS do anything in that situation?
In short, yes. If your MLS has a RETS server that you have access to, you could use any RETS-compliant program (called RETS clients) to access that data. This introduces more options for agents and brokers because they aren't tied to single user interface.
There are drawbacks, however. In most cases, the MLS software offers a lot of features. So far, it's difficult for a RETS client to offer the same features that the MLS might be able to offer through their own software because their software is made specifically to work with their data in their area. Work is continuing with RETS to try and introduce more capabilities for information sharing.
(continued in Part 6)
RETS, the Real Estate Transaction Standard, is our solution.
This standard allows the MLS's to meet you halfway. Maybe they won't give you direct access to their database but you can have live access to their database through a protected system interface.
Now we're talking.
As you can probably tell by the name, RETS is the standard used to move real estate information back and forth. The idea is that RETS defines:
Maybe you would normally download the once-daily file from an FTP server somewhere. 1 MLS you work with provides it in the /IDX/download/daily directory with a filename like "listings-residential.tar.gz" and another MLS provides it in the /NEW_IDX directory with a filename like "PropResi-5_11_2007-06_27_01-full.zip". With RETS, you connect and communicate in a common way so you don't have to deal with unique situations like this. RETS replaces the need for an FTP server with a completely separate authentication system. In addition, because your client RETS program logs into a server RETS program, the server can apply business-specific rules and restrictions on you which might give you more access to more data than someone else might have.
Like we mentioned before, the data you normally get could be in any format with any number of fields with any kind of data type (text, numbers, etc.). With RETS, the format of the data is available in a live, technically accessible format so you know exactly what you're going to get. If the MLS adds some new fields to the feed, information about those fields is available so you can make the necessary changes on your end (hopefully using a program that does it automatically).
If this wasn't enough, RETS also defines common names for fields called StandardNames. This allows more standardization among all of the MLSs that support RETS. Normally, one MLS might provide you with a field called MLS_Number and another might give you MLNUM. These are the same fields but they have different labels. With RETS using StandardNames, you can know and expect that ListingID will be the MLS# for that property. This allows for much greater interoperability so you can use the same RETS client with any RETS server and have the same results. Try that with your FTP downloader!
(continued in Part 5)
This standard allows the MLS's to meet you halfway. Maybe they won't give you direct access to their database but you can have live access to their database through a protected system interface.
Now we're talking.
As you can probably tell by the name, RETS is the standard used to move real estate information back and forth. The idea is that RETS defines:
- how the 2 systems (client-server) connect and communicate back and forth
- the format of the information being passed back and forth
Maybe you would normally download the once-daily file from an FTP server somewhere. 1 MLS you work with provides it in the /IDX/download/daily directory with a filename like "listings-residential.tar.gz" and another MLS provides it in the /NEW_IDX directory with a filename like "PropResi-5_11_2007-06_27_01-full.zip". With RETS, you connect and communicate in a common way so you don't have to deal with unique situations like this. RETS replaces the need for an FTP server with a completely separate authentication system. In addition, because your client RETS program logs into a server RETS program, the server can apply business-specific rules and restrictions on you which might give you more access to more data than someone else might have.
Like we mentioned before, the data you normally get could be in any format with any number of fields with any kind of data type (text, numbers, etc.). With RETS, the format of the data is available in a live, technically accessible format so you know exactly what you're going to get. If the MLS adds some new fields to the feed, information about those fields is available so you can make the necessary changes on your end (hopefully using a program that does it automatically).
If this wasn't enough, RETS also defines common names for fields called StandardNames. This allows more standardization among all of the MLSs that support RETS. Normally, one MLS might provide you with a field called MLS_Number and another might give you MLNUM. These are the same fields but they have different labels. With RETS using StandardNames, you can know and expect that ListingID will be the MLS# for that property. This allows for much greater interoperability so you can use the same RETS client with any RETS server and have the same results. Try that with your FTP downloader!
(continued in Part 5)
Getting a little more technical now.....
Let's assume you're developing a website for someone. Naturally, the best way to have updated information in a solid, technically noticeable format is to have direct access to the MLS's database. If you're betting on the MLS giving this to you, I think you'll be disappointed.
Furthermore, consider the extremely possible scenario where you're running one type of system and the MLS is running another. Trying to simply copy their data into your format is trying to fit a square peg into a round hole. You can't do it without a lot of work and finesse.
Maybe you get "BedsAll BathsAll StreetAddress BathsPartial RoomsAll" from the MLS (maybe an Oracle database) and you're trying to fit that into a "Bedrooms TotalRooms FullBathrooms HalfBathrooms StreetNumber StreetDirection StreetName" format on your system (maybe a MySQL database).
Square peg, round hole.
So...
Direct access to a database is too much. Access to an FTP server somewhere with once-daily updated data isn't enough.
Meet RETS.
(continued in Part 4)
Let's assume you're developing a website for someone. Naturally, the best way to have updated information in a solid, technically noticeable format is to have direct access to the MLS's database. If you're betting on the MLS giving this to you, I think you'll be disappointed.
Furthermore, consider the extremely possible scenario where you're running one type of system and the MLS is running another. Trying to simply copy their data into your format is trying to fit a square peg into a round hole. You can't do it without a lot of work and finesse.
Maybe you get "BedsAll BathsAll StreetAddress BathsPartial RoomsAll" from the MLS (maybe an Oracle database) and you're trying to fit that into a "Bedrooms TotalRooms FullBathrooms HalfBathrooms StreetNumber StreetDirection StreetName" format on your system (maybe a MySQL database).
Square peg, round hole.
So...
Direct access to a database is too much. Access to an FTP server somewhere with once-daily updated data isn't enough.
Meet RETS.
(continued in Part 4)
So, now, you've got data somewhere online in an exported format from the MLS. Great! That just saved a large amount of time, but we aren't done yet.
The good:
The bad:
(Note that everything above is just talking about getting and processing the data portion of the feed. Getting and processing photos for those properties is normally a totally different process with it's own challenges. For the sake of keeping this simple, we'll ignore this aspect of it for now.)
If you're shopping around the Internet for an IDX vendor for your IDX-enabled website so you can show site users properties from your local MLS, the above process is the challenge. If you're the IDX vendor, you're all too familiar with this process because you probably manage dozens of data feeds and experience these kinds of problems almost on a daily basis. My heart goes out to you.
So, what can we do to make this even better? This is typically where you stop if you've never heard of RETS, but we're not stopping because you're about to.
(continued in Part 3)
The good:
- You can actually see information about properties!
- The MLS didn't have to do much to make that happen because they probably allow many people to have access to that information so you probably aren't charged anything (or at least, not as much) to make that happen. Saved money!
- They automatically update the data every day (usually) so you can expect updated property data that often. Sure beats copy/pasting information every day.
The bad:
- You've got a lot of information but it's hard to read. You need to figure out how to break the file apart so you can know what each piece of data means. Not too bad.
- You've got a file in a semi-programmable format but you don't know what fields are where and what each piece of information is. In most cases, the information for actually understanding the file is available in PDF format from your MLS. Having this documentation is extremely helpful but isn't updated often and it's in a format that you can't have a program understand so you end up reading it and interpreting it manually. Yuck.
- Data that's updated daily is decent in most cases, but what if you want something a little more often? You could download the file every 60 seconds if you wanted but unless it's updated, it doesn't do anything for you. You'd need the MLS's cooperation for that. Probably not going to happen.
- When the MLS makes changes to the data (i.e. they add a new field to the feed), there's very little you can do to know about this. If they don't tell you ahead of time, you've just been blindsided. You're expecting 150 pieces of information and you get 151 so things start to break down. You can't see what changed because, chances are, you don't get a new PDF every evening and if you request it from them again, it's probably not updated.
(Note that everything above is just talking about getting and processing the data portion of the feed. Getting and processing photos for those properties is normally a totally different process with it's own challenges. For the sake of keeping this simple, we'll ignore this aspect of it for now.)
If you're shopping around the Internet for an IDX vendor for your IDX-enabled website so you can show site users properties from your local MLS, the above process is the challenge. If you're the IDX vendor, you're all too familiar with this process because you probably manage dozens of data feeds and experience these kinds of problems almost on a daily basis. My heart goes out to you.
So, what can we do to make this even better? This is typically where you stop if you've never heard of RETS, but we're not stopping because you're about to.
(continued in Part 3)
For a moment, forget about protocols, character lengths, abstraction layers, integer fields, HTTP headers and all of that, and let's look at the problem starting from the very beginning:
If you're an agent or broker (or a developer for one), the MLS has data and you want it.
They could print off each property sheet and mail it to you but that's too slow and inefficient and doesn't allow you to really do anything with that information once you have it.
They could call you over the phone and read off everything to someone that would re-type it all in a format that you can use, but that takes even longer and requires phone time with people that would probably takes weeks to get that information, and by the time you're done, it's out of date.
They could copy/paste the property sheets into emails and email them to you. Even though this is a little faster, that still turns into a TON of emails and you'd have to copy/paste that information into something that makes sense to you.
You could copy/paste the property sheets using your own MLS access software but that's really sloppy, takes a ton of time and still doesn't get you the information you want so you can use it without a lot more work.
Let's get a little more technical.....
They could create an exported version of the MLS data and make it available somewhere online. You'd somehow grab that data, figure out what it all meant (what format it was in, how things were separated, etc.) and then turn it into something you could use. This is a much better solution compared to the ones above but still has a few drawbacks. We're getting close, though.
(continued in Part 2)
If you're an agent or broker (or a developer for one), the MLS has data and you want it.
They could print off each property sheet and mail it to you but that's too slow and inefficient and doesn't allow you to really do anything with that information once you have it.
They could call you over the phone and read off everything to someone that would re-type it all in a format that you can use, but that takes even longer and requires phone time with people that would probably takes weeks to get that information, and by the time you're done, it's out of date.
They could copy/paste the property sheets into emails and email them to you. Even though this is a little faster, that still turns into a TON of emails and you'd have to copy/paste that information into something that makes sense to you.
You could copy/paste the property sheets using your own MLS access software but that's really sloppy, takes a ton of time and still doesn't get you the information you want so you can use it without a lot more work.
Let's get a little more technical.....
They could create an exported version of the MLS data and make it available somewhere online. You'd somehow grab that data, figure out what it all meant (what format it was in, how things were separated, etc.) and then turn it into something you could use. This is a much better solution compared to the ones above but still has a few drawbacks. We're getting close, though.
(continued in Part 2)
