The Twitter API was one of the first available from a major Internet organization and it is probably the most used API worldwide, even though in the past few years Twitter has been making it tougher for developers to use it, you may check this as an example.
Not everybody is on Twitter but almost every company is, specially big telcos. Most of them even provide customer support through Twitter, and despite being usually terrible (the support), at least it is an open channel and you can share your frustration with other frustrated fellow customers.
I've got very mad, Jazztel (the telco) was not interested in finding the truth whatsoever, they were accusing my mom of not paying her bills and threatening her to go to court, this telephonic conversation (in Spanish) proves it.
I've got inspiration from this post which I read a few months ago.
But my "avenger bot" is more sophisticated than that, it uses the Twitter streaming API to listen for tweets addressed to Jazztel and then reply to them with an angry complain (I am aware that it probably won't make any difference but at least it was a way to release my anger while learning something since it was technically challenging and I also found it interesting).
Luckily I had all that from working on a previous pet project of mine, because since few months ago, Twitter forces new users of their API to register for a developer account and wait for the registration to (hopefully) be approved.
Then I started working on my bot, and after few evenings I had it ready, the whole source code (Java 8) is available on GitHub.
These are the main features:
- The four security keys:
Consumer Key (API Key)
Consumer Secret (API Secret)
Access Token
Access Token Secret
are encrypted in the JSON configuration file and decrypted at runtime using a password stored in an environment variable.
- It uses very simple dependency injection through Spring Framework, I chose XML configuration over annotations because as Sam Atkinson said in his famous post more than four and a half years ago: I don't like magic.
I do not like to "code" in XML either, that is why I said "very simple dependency injection" before, I strongly believe that Spring (or any other DI framework for that matter) should be used just to wire beans together, nothing more, no smart/custom type conversions, no SpEL, etc...
- It implements the producer/consumer pattern using a blocking queue.
- The application properly shuts down (disconnecting gracefully from the stream of tweets) on Linux after receiving the WINCH signal by implementing the SignalHandler interface (the complete motivation and explanation of this shutdown mechanism deserves its own post which I may write in the near future).
- The bot uses Twitter4j library so I do not need to deal directly with REST/HTTP.
- There are no unit tests but at least some "tester" classes are included which helped me to validate the code as I was writing it.
The first time I started it, everything seemed to be working fine, every time a new tweet containing the keywords defined in the config.json was published, the bot would reply after waiting a semi-random amount of time.
But after an hour or so, I got the following ugly error:
Which meant that my application had been banned from publishing tweets, even though it could still read them.
I could see it also in the Twitter Web UI
But why? I had been cautious not to reach the Twitter API limits, my bot always waited for more than 2 and a half minutes between two consecutive tweet submissions, much longer than the 36 seconds which seems to be the limit.
After some research I found the answer in the Twitter spamming rules:
My bot randomly selects the content of the tweets from a pool of twenty texts, so after several tweets were published containing the same text, my application was banned.
In this case, several unanswered questions come to my mind:
What exactly makes two tweets "substantially similar"?
If two tweets contain the same words but the order is different does Twitter consider them duplicates?
What if only 80% of the words are the same? And what if 60%?
Is it ok to repeat the same tweet just once a day? Or maybe once a week?
The Twitter documentation intentionally does not answer these questions and the information available from other sources is quite ambiguous.
So my only option is experimentation and testing.
Does this mean that all the hassle was in vain? Absolutely not!
I had an idea, a challenge, and I went through with it.
Something bothered me and I did something about it (and I also enjoyed it).
So even though the result is not ideal (yet) because of some unforeseen limitation, the whole experience was totally worth it. I have got to experiment with something new to me and I also get the chance to share the whole episode with whoever my be interested in it.
Currently I am waiting for my Twitter developer request to be accepted so that I can create a new app. Once I am able to publish tweets using the API again, I may even do some research and throw in some AI in order to generate distinct enough tweets to be published without Twitter banning me...
So if I find a workaround, I may also launch my Avenger Bot against another company which disrespects their customer/passengers: SmartWings.
Stay tuned!
Not everybody is on Twitter but almost every company is, specially big telcos. Most of them even provide customer support through Twitter, and despite being usually terrible (the support), at least it is an open channel and you can share your frustration with other frustrated fellow customers.
Motivation
So when one of the major telecommunications operators in Spain started aggressively chasing my mom and my whole family because someone used her name and national identity number to fraudulently buy services from them which were never paid (it seems they intentionally? have barely any security controls in place), I decided to take revenge on Twitter.I've got very mad, Jazztel (the telco) was not interested in finding the truth whatsoever, they were accusing my mom of not paying her bills and threatening her to go to court, this telephonic conversation (in Spanish) proves it.
I've got inspiration from this post which I read a few months ago.
But my "avenger bot" is more sophisticated than that, it uses the Twitter streaming API to listen for tweets addressed to Jazztel and then reply to them with an angry complain (I am aware that it probably won't make any difference but at least it was a way to release my anger while learning something since it was technically challenging and I also found it interesting).
Implementation
So the first thing I needed was a Twitter account under which my bot would run and then also a "Twitter app" in order to retrieve the four security keys required for authentication and authorization against the Twitter API.Luckily I had all that from working on a previous pet project of mine, because since few months ago, Twitter forces new users of their API to register for a developer account and wait for the registration to (hopefully) be approved.
Then I started working on my bot, and after few evenings I had it ready, the whole source code (Java 8) is available on GitHub.
These are the main features:
- The four security keys:
Consumer Key (API Key)
Consumer Secret (API Secret)
Access Token
Access Token Secret
are encrypted in the JSON configuration file and decrypted at runtime using a password stored in an environment variable.
- It uses very simple dependency injection through Spring Framework, I chose XML configuration over annotations because as Sam Atkinson said in his famous post more than four and a half years ago: I don't like magic.
I do not like to "code" in XML either, that is why I said "very simple dependency injection" before, I strongly believe that Spring (or any other DI framework for that matter) should be used just to wire beans together, nothing more, no smart/custom type conversions, no SpEL, etc...
- It implements the producer/consumer pattern using a blocking queue.
- The application properly shuts down (disconnecting gracefully from the stream of tweets) on Linux after receiving the WINCH signal by implementing the SignalHandler interface (the complete motivation and explanation of this shutdown mechanism deserves its own post which I may write in the near future).
- The bot uses Twitter4j library so I do not need to deal directly with REST/HTTP.
- There are no unit tests but at least some "tester" classes are included which helped me to validate the code as I was writing it.
Execution
After I completed the coding, I was ready to run it and see the bot in action.The first time I started it, everything seemed to be working fine, every time a new tweet containing the keywords defined in the config.json was published, the bot would reply after waiting a semi-random amount of time.
But after an hour or so, I got the following ugly error:
Which meant that my application had been banned from publishing tweets, even though it could still read them.
I could see it also in the Twitter Web UI
But why? I had been cautious not to reach the Twitter API limits, my bot always waited for more than 2 and a half minutes between two consecutive tweet submissions, much longer than the 36 seconds which seems to be the limit.
After some research I found the answer in the Twitter spamming rules:
My bot randomly selects the content of the tweets from a pool of twenty texts, so after several tweets were published containing the same text, my application was banned.
Future work
Something that all software developers hate are loose requirements but they are everyday fare and whole methodologies such as Scrum have been created to tackle this issue.In this case, several unanswered questions come to my mind:
What exactly makes two tweets "substantially similar"?
If two tweets contain the same words but the order is different does Twitter consider them duplicates?
What if only 80% of the words are the same? And what if 60%?
Is it ok to repeat the same tweet just once a day? Or maybe once a week?
The Twitter documentation intentionally does not answer these questions and the information available from other sources is quite ambiguous.
So my only option is experimentation and testing.
Does this mean that all the hassle was in vain? Absolutely not!
I had an idea, a challenge, and I went through with it.
Something bothered me and I did something about it (and I also enjoyed it).
So even though the result is not ideal (yet) because of some unforeseen limitation, the whole experience was totally worth it. I have got to experiment with something new to me and I also get the chance to share the whole episode with whoever my be interested in it.
Currently I am waiting for my Twitter developer request to be accepted so that I can create a new app. Once I am able to publish tweets using the API again, I may even do some research and throw in some AI in order to generate distinct enough tweets to be published without Twitter banning me...
So if I find a workaround, I may also launch my Avenger Bot against another company which disrespects their customer/passengers: SmartWings.
Stay tuned!
Comments
Post a Comment