19 replies to this topic

[Xerax]

So for the past month I've been working with a friend to look into the Halo 4 Stats website on http://app.halowaypoint.com/. And finally, after a lot of work, we're ready to release the Source Code, how to access and use the internal API and a small example website to show off basics of what can be done.


Developers
- We have a wiki of information on how to Authenticate Endpoints, and send requests
- We have examples on how to use the API

Users
- We have a couple of examples that are ready for users to use
- http://auntiedot.net/ - http://leafapp.co/

For more information, visit: http://auntiedot.net

(Also, if you could upvote it on reddit (http://www.reddit.co...th_api_finally/) that would be awesome, so that the Halo Community can start using it.

[EatonZ]

An internal API? Interesting.. nice discover/SDK.

[iLoch]

Really? You're going to take credit for this? I, and many others, have had access to this API since it launched on November 4th. We've specifically avoided releasing the information to the public because people do not understand how to use APIs. Now - thanks to your efforts - the API will be restricted as it get's slammed by sloppy programmers with useless applications. I've been working with HaloTracker in order to get stats working on that site (which has over 100,000 registered gamertags) and now they're at risk of being blocked because of other people abusing the API.

Please be more responsible next time you decide to release a private API. Do so from a closed environment. Provide developers with access through a proxy, not direct access. Say goodbye to the API.

[Xerax]

iLoch, on 05 February 2013 - 19:05, said:

Really? You're going to take credit for this? I, and many others, have had access to this API since it launched on November 4th. We've specifically avoided releasing the information to the public because people do not understand how to use APIs. Now - thanks to your efforts - the API will be restricted as it get's slammed by sloppy programmers with useless applications. I've been working with HaloTracker in order to get stats working on that site (which has over 100,000 registered gamertags) and now they're at risk of being blocked because of other people abusing the API.

Please be more responsible next time you decide to release a private API. Do so from a closed environment. Provide developers with access through a proxy, not direct access. Say goodbye to the API.

Sorry, did we knock you from your Ivory Tower? Some people believe sharing is caring. And lets face it, 343 know people have been doing this. If they cared, they would of taken action already.

[iLoch]

Xerax, on 05 February 2013 - 19:07, said:

Sorry, did we knock you from your Ivory Tower? Some people believe sharing is caring. And lets face it, 343 know people have been doing this. If they cared, they would of taken action already.

The most responsible developers in the community have had access for a while. We know sharing is caring. There's a right, and a wrong way of doing it. (See http://halocharts.com JSON service) By releasing this source code, you've allowed anyone to hit the service with a million requests. 343i already have tightened the API - my guess is that you didn't notice since you didn't get access right away. Next step is an IP whitelist.

[Xerax]

iLoch, on 05 February 2013 - 19:13, said:

The most responsible developers in the community have had access for a while. We know sharing is caring. There's a right, and a wrong way of doing it. (See http://halocharts.com JSON service) By releasing this source code, you've allowed anyone to hit the service with a million requests. 343i already have tightened the API - my guess is that you didn't notice since you didn't get access right away. Next step is an IP whitelist.

IP Whitelist? Please. Stop.

[Hardcore Til I Die]

iLoch, on 05 February 2013 - 19:05, said:

Really? You're going to take credit for this? I, and many others, have had access to this API since it launched on November 4th. We've specifically avoided releasing the information to the public because people do not understand how to use APIs. Now - thanks to your efforts - the API will be restricted as it get's slammed by sloppy programmers with useless applications. I've been working with HaloTracker in order to get stats working on that site (which has over 100,000 registered gamertags) and now they're at risk of being blocked because of other people abusing the API.

Please be more responsible next time you decide to release a private API. Do so from a closed environment. Provide developers with access through a proxy, not direct access. Say goodbye to the API.

With all due respect they MUST have expected this to happen.

They can't release an API to a bunch of people outside their organisation and expect it not to be passed around.

[iLoch]

Xerax, on 05 February 2013 - 19:52, said:

IP Whitelist? Please. Stop.

Sorry, is that hard for you to believe? That's actually just one line they would have to add to block 100% of the unwanted access. Are you kidding me?

[BillinghamJ]

FYI - I'm the other developer/researcher of this stuff.

iLoch, on 05 February 2013 - 19:13, said:

The most responsible developers in the community have had access for a while. We know sharing is caring. There's a right, and a wrong way of doing it. (See http://halocharts.com JSON service) By releasing this source code, you've allowed anyone to hit the service with a million requests. 343i already have tightened the API - my guess is that you didn't notice since you didn't get access right away. Next step is an IP whitelist.

Do you actually know what you're talking about?

It is a JSON API, which is accessed by the HWP website through AJAX. Each user makes their own requests to it - therefore IP whitelisting would be impossible.

The security on it is extremely basic at present, so I'm not sure what you mean by "343i already have tightened the API" since it's really not in the slightest bit difficult to authenticate to it.

An intermediate service (rather than documenting how to access directly) would be an incredibly stupid thing to do for several reasons:

• It would be painfully obvious since a ridiculous number of requests to different accounts would be coming from the same IP
  
• It would be incredibly easy to block
  
• It would require working around, rather than with, the authentication system. Our docs & client conform fully to the way the auth system was intended to work and, as such, there are no security issues in terms of phishing & no reason for 343/MS to get ****ed off about it

If anyone does use our library to send millions of requests or make an intermediate service, they're an idiot, but luckily it's easy for 343 to block. The research/library should only be used (& will only work long term) as part of a client application executing on the user's computer - whether that be with JavaScript or a compiled app.

Additionally, this is not an internal API. A private one, maybe, but every time you visit HWP you directly call it 20-100 times (ish), so it's not exactly hidden.

iLoch, on 05 February 2013 - 20:03, said:

Sorry, is that hard for you to believe? That's actually just one line they would have to add to block 100% of the unwanted access. Are you kidding me?

No, no. Are you kidding me? **** off and come back when you actually know what AJAX is and how you build a Javascript based client side web app.

[iLoch]

Hardcore Til I Die, on 05 February 2013 - 19:58, said:

With all due respect they MUST have expected this to happen.

They can't release an API to a bunch of people outside their organisation and expect it not to be passed around.

Well they're using the best authentication they can get - so my guess is that they're trying to provide their users with the best possible experience at the expense of data security. Now they're just going to clamp down on it. It's nothing to them. Of course they expected it to happen - but they weren't going to do anything about it if it was just a few community sites accessing the information. Now this is a real problem, and they'll find a solution.

[BillinghamJ]

iLoch, on 05 February 2013 - 20:06, said:

Well they're using the best authentication they can get - so my guess is that they're trying to provide their users with the best possible experience at the expense of data security. Now they're just going to clamp down on it. It's nothing to them. Of course they expected it to happen - but they weren't going to do anything about it if it was just a few community sites accessing the information. Now this is a real problem, and they'll find a solution.

Tbh, I don't think they're bothered. The programme manager of the Halo web team followed Xerax & I on Twitter. I messaged him letting him know we're more than happy to work with them if they don't like what we're doing or want us to do something for them.

[iLoch]

BillinghamJ, on 05 February 2013 - 20:06, said:

FYI - I'm the other developer/researcher of this stuff.



Do you actually know what you're talking about?

It is a JSON API, which is accessed by the HWP website through AJAX. Each user makes their own requests to it - therefore IP whitelisting would be impossible.

The security on it is extremely basic at present, so I'm not sure what you mean by "343i already have tightened the API" since it's really not in the slightest bit difficult to authenticate to it.

An intermediate service (rather than documenting how to access directly) would be an incredibly stupid thing to do for several reasons:

• It would be painfully obvious since a ridiculous number of requests to different accounts would be coming from the same IP
  
• It would be incredibly easy to block
  
• It would require working around, rather than with, the authentication system. Our docs & client conform fully to the way the auth system was intended to work and, as such, there are no security issues in terms of phishing & no reason for 343/MS to get ****ed off about it

If anyone does use our library to send millions of requests or make an intermediate service, they're an idiot, but luckily it's easy for 343 to block. The research/library should only be used (& will only work long term) as part of a client application executing on the user's computer - whether that be with JavaScript or a compiled app.

Additionally, this is not an internal API. A private one, maybe, but every time you visit HWP you directly call it 20-100 times (ish), so it's not exactly hidden.



No, no. Are you kidding me? **** off and come back when you actually know what AJAX is and how you build a Javascript based client side web app.

Alright, so I guess the fact that I have been doing this for seven years doesn't mean anything. Apparently you don't understand AJAX. You cannot make requests to the API clientside from your users' computer (through a web browser), because of cross-origin access policies. You are forced to send connections through your server, which means it's one IP to block all requests you want to make from your site. What you're describing, and the API you provided, works great for desktop applications, but will completely **** over websites like HaloTracker, which is where the majority of the calls are coming from, not desktop applications.

And as I've already stated, I had access to this API the same day it was released and was able to fully authenticate without problems. When it was first released, they did not require any data-scraping, it was 100% header based. They've locked it down more by embedding it within the page.

[BillinghamJ]

iLoch, on 05 February 2013 - 20:12, said:

Alright, so I guess the fact that I have been doing this for seven years doesn't mean anything. Apparently you don't understand AJAX. You cannot make requests to the API clientside from your users' computer, because of cross-origin access policies. You are forced to send connections through your server, which means it's one IP to block all requests you want to make from your site. What you're describing, and the API you provided, works great for desktop applications, but will completely **** over websites like HaloTracker, which is where the majority of the calls are coming from, not desktop applications.

And as I've already stated, I had access to this API the same day it was released and was able to fully authenticate without problems. When it was first released, they did not require any data-scraping, it was 100% header based. They've locked it down more by embedding it within the page.

'Tis a good point which I hadn't considered, but regardless, going through a server is never going to be a sustainable way of accessing an API like this.

Also, I've been doing this for more than seven years, so

[iLoch]

BillinghamJ, on 05 February 2013 - 20:21, said:

'Tis a good point which I hadn't considered, but regardless, going through a server is never going to be a sustainable way of accessing an API like this.

Also, I've been doing this for more than seven years, so

Not long enough then if you hadn't considered that, so... And for most uses, that's the only way of accessing the API, which is why everyone with a big site is extremely ticked off at you two right now.

[BillinghamJ]

iLoch, on 05 February 2013 - 20:36, said:

Not long enough then if you hadn't considered that, so... And for most uses, that's the only way of accessing the API, which is why everyone with a big site is extremely ticked off at you two right now.

To be fair, anyone with a big site wouldn't have lasted long anyway. Us doing this is unlikely to make much difference.