Pushed a new snapshot

Support for Twitter video playback

.gitignore

@@ -20,3 +20,4 @@ release-snapshot/
 src/com_cache/
 doc/strings.py
 doc/changelog.py
+env/

README.md

@@ -24,45 +24,37 @@ This document describes how to run tw blue from source and how to build a binary
 ### Required dependencies.
 
 Although most dependencies can be found in the windows-dependencies directory, we provide links to their official websites. If you are cloning with git, don't forget to initialize and update the submodules to get the windows-dependencies folder. You can use these two commands to perform this task from git bash:  
+```
     git submodule init  
     git submodule update
-
-	All the dependencies provided in this folder are prebuilt. If you want to build them from source, you will need Microsoft visual Studio 2008.
+```
 
 #### Dependencies packaged in windows installers
 
-* [Python,](http://python.org) version 2.7.16  
-If you want to build both x86 and x64 binaries, you can install python x86 to C:\python27 and python x64 to C:\python27x64, for example.
-* [PyEnchant,](http://pythonhosted.org/pyenchant/) version 1.6.6.  
-x64 version has been built by TWBlue developers, so you only will find it in windows-dependencies folder
-
-The windows installers are available only in the windows-dependencies folder.
-
-To build a binary version:
-
-* [Py2exe](http://www.sourceforge.net/projects/py2exe/) for Python 2.7, version 0.6.9
+* [Python,](https://python.org) version 3.8.7  
+If you want to build both x86 and x64 binaries, you can install python x86 to C:\python38 and python x64 to C:\python38x64, for example.
 
 #### Dependencies that must be installed using pip
 
 Python installs a tool called Pip that allows to install packages in a simple way. You can find it in the python scripts directory. To install packages using Pip, you have to navigate to the scripts directory using a command prompt, for example:
 
-    cd C:\python27x64\scripts
+    `cd C:\python37x64\scripts`
 
 	You can also add the scripts folder to your path environment variable or choose the corresponding option when installing Python.  
 	Note: pip and setuptools are included in the Python installer since version 2.7.9.
 
 Pip is able to install packages listed in a special text file, called the requirements file. To install all remaining dependencies, perform the following command:
 
-    pip install -r requirements.txt
+    `pip install -r requirements.txt`
 
 Note that if you perform the command from the path where Pip is located, you need to specify the path to your Tw Blue root folder where the requirements file is located, for example:
 
-    pip install -r D:\repos\TwBlue\requirements.txt
+    `pip install -r D:\repos\TwBlue\requirements.txt`
 
 Pip will automatically get the additional libraries that the listed packages need to work properly.  
 If you need to update your dependencies, perform the following command:
 
-    pip install --upgrade -r requirements.txt
+    `pip install --upgrade -r requirements.txt`
 
 #### Other dependencies
 
@@ -71,7 +63,7 @@ These dependencies are located in the windows-dependencies directory. You don't
 * Bootstrap 1.2.1: included in dependencies directory.  
 This dependency has been built using pure basic 4.61. Its source can be found at http://hg.q-continuum.net/updater
 * [oggenc2.exe,](http://www.rarewares.org/ogg-oggenc.php) version 2.87  
-* Microsoft Visual c++ 2008 redistributable dlls.
+* Microsoft Visual c++ 2019 redistributable dlls.
 * VLC plugins and DLL libraries.
 
 #### Dependencies required to build the installer
@@ -94,7 +86,7 @@ In order to add the support for spell checking in more languages than english yo
 
 Now that you have installed all these packages, you can run TW Blue from source using a command prompt. Navigate to the repo's `src` directory, and type the following command:
 
-    python main.py
+    `python main.py`
 
 	If necessary, change the first part of the command to reflect the location of your python executable. You can run TW Blue using python x86 and x64.
 
@@ -102,8 +94,8 @@ Now that you have installed all these packages, you can run TW Blue from source
 
 To generate the documentation in html format, navigate to the doc folder inside this repo. After that, run these commands:  
 
-    python document_importer.py  
-    python generator.py  
+    `python document_importer.py`  
+    `python generator.py`  
 
 The documentation will be generated, placing each language in a separate folder in the doc directory. Move these folders (for example `de`, `en`, `es`, `fr`, `it`, ...) to `src/documentation`, creating the directory if necessary.  
 Also, copy the `license.txt` file located in the root of the repo to the documentation folder.
@@ -114,7 +106,7 @@ A binary version doesn't need python and the other dependencies to run, it's the
 
 To build it, run the following command from the src folder:
 
-    python setup.py py2exe
+    `python setup.py build`
 
 	You will find the binaries in the dist directory.
 
@@ -122,9 +114,9 @@ To build it, run the following command from the src folder:
 
 If you want to install TWBlue on your computer, you must create the installer first. Follow these steps:
 
-* Navigate to the src directory, and create a binary version for x86: C:\python27\python setup.py py2exe
+* Navigate to the src directory, and create a binary version for x86: C:\python37\python setup.py build
 * Move the dist directory to the scripts folder in this repo, and rename it to twblue
-* Repeat these steps with Python for x64: C:\python27x64\python setup.py py2exe
+* Repeat these steps with Python for x64: C:\python37x64\python setup.py build
 * Move the new dist directory to the scripts folder, and rename it to twblue64
 * Go to the scripts folder, right click on the twblue.nsi file, and choose compyle unicode NSIS script
 * This may take a while. After the process, you will find the installer in the scripts folder
@@ -137,9 +129,9 @@ Run the gen_pot.bat file, located in the tools directory. Your python installati
 
 If you want to have TWBlue on your PortableApps.com platform, follow these steps:
 
-* Navigate to the src directory, and create a binary version for x86: C:\python27\python setup.py py2exe
+* Navigate to the src directory, and create a binary version for x86: C:\python37\python setup.py build
 * Move the dist directory to the misc\pa.c format\app folder in this repo, and rename it to twblue
-* Repeat these steps with Python for x64: C:\python27x64\python setup.py py2exe
+* Repeat these steps with Python for x64: C:\python37x64\python setup.py build
 * Move the new dist directory to the misc\pa.c format\app folder, and rename it to twblue64
 * Run the PortableApps.com Launcher Generator, and follow the wizard. Choose the pa.c format folder and continue to generate the launcher. If the wizard is completed, you will see a file named TWBlue portable.exe inside the pa.c format folder.
 * Run the PortableApps.com Installer, and follow the wizard. As in the above step, choose the pa.c format folder. When it completes, you will see a file named TWBluePortable_x.y.paf.exe inside the misc folder, where x.y is the version number.

appveyor.yml

@@ -10,8 +10,8 @@ environment:
   matrix:
 
   # List of python versions we want to work with.
-    - PYTHON: "C:\\Python37"
-      PYTHON_VERSION: "3.7.x" # currently 2.7.9
+    - PYTHON: "C:\\Python38"
+      PYTHON_VERSION: "3.8.x" # currently 2.7.9
       PYTHON_ARCH: "32"
 
   # perhaps we may enable this one in future?

contributors.txt

@@ -39,4 +39,5 @@ florian Ionașcu
 Christian Leo Mameli
 Natalia Hedlund (Наталья Хедлунд)
 Valeria (Валерия)
-Corentin Bacqué-Cazenave
+Oreonan
+Artem Plaksin (maniyax)

doc/changelog.md

@@ -2,7 +2,30 @@
 
 ## changes in this version
 
-* Fixed error when displaying an URL at the end of a line, when the tweet or direct message contained multiple lines. Now the URL should be displayed correctly. ((#305,)[https://github.com/manuelcortez/TWBlue/issues/305])
+* Added support for Twitter audio and videos: Tweets which contains audio or videos will be detected as audio items, and you can playback those with the regular command to play audios. ([#384,](https://github.com/manuelcortez/TWBlue/pull/384))
+* We just implemented some changes in the way TWBlue handles tweets in order to reduce its RAM memory usage [#380](https://github.com/manuelcortez/TWBlue/pull/380):
+    * We reduced the tweets size by storing only the tweet fields we currently use. This should reduce tweet's size in memory for every object up to 75%.
+    * When using the cache database to store your tweets, there is a new setting present in the account settings dialog, in the general tab. This setting allows you to control whether TWBlue will load the whole database into memory (which is the current behaviour) or not.
+        * Loading the whole database into memory has the advantage of being extremely fast to access any element (for example when moving through tweets in a buffer), but it requires more memory as the tweet buffers grow up. This should, however, use less memory than before thanks to the optimizations performed in tweet objects. If you have a machine with enough memory, this should be a good option for your case.
+        * If you uncheck this setting, TWBlue will read the whole database from disk. This is significantly slower, but the advantage of this setting is that it will consume almost no extra memory, no matter how big is the tweets dataset. Be ware, though, that TWBlue might start to feel slower when accessing elements (for example when reading tweets) as the buffers grow up. This setting is suggested for computers with low memory or for those people not wanting to keep a really big amount of tweets stored.
+* Changed the label in the direct message's text control so it will indicate that the user needs to write the text there, without referring to any username in particular. ([#366,](https://github.com/manuelcortez/TWBlue/issues/366))
+* TWBlue will take Shift+F10 again as the contextual menu key in the list of items in a buffer. This stopped working after we have migrated to WX 4.1. ([#353,](https://github.com/manuelcortez/TWBlue/issues/353))
+* TWBlue should render correctly retweets of quoted tweets. ([#365,](https://github.com/manuelcortez/TWBlue/issues/365))
+* Fixed an error that was causing TWBlue to be unable to output to screen readers at times. ([#369,](https://github.com/manuelcortez/TWBlue/issues/369))
+* Fixed autocomplete users feature. ([#367,](https://github.com/manuelcortez/TWBlue/issues/367))
+* Fixed error when displaying an URL at the end of a line, when the tweet or direct message contained multiple lines. Now the URL should be displayed correctly. ([#305,](https://github.com/manuelcortez/TWBlue/issues/305) [#272,](https://github.com/manuelcortez/TWBlue/issues/272))
+* TWBlue has been migrated completely to Python 3 (currently, the software builds with Python 3.8).
+* TWBlue should be restarted gracefully. Before, the application was alerting users of not being closed properly every time the application restarted by itself.
+* If TWBlue attemps to load an account with invalid tokens (this happens when reactivating a previously deactivated account, or when access to the ap is revoqued), TWBlue will inform the user about this error and will skip the account. Before, the app was unable to start due to a critical error. ([#328,](https://github.com/manuelcortez/TWBlue/issues/328))
+* When sending a direct message, the title of the window will change appropiately when the recipient is edited. ([#276,](https://github.com/manuelcortez/TWBlue/issues/276))
+* URL'S in user profiles are expanded automatically. ([#275,](https://github.com/manuelcortez/TWBlue/issues/275))
+* TWBlue now uses [Tweepy,](https://github.com/tweepy/tweepy) to connect with Twitter. We have adopted this change in order to support Twitter'S API V 2 in the very near future. ([#333,](https://github.com/manuelcortez/TWBlue/issues/337) [#347](https://github.com/manuelcortez/TWBlue/pull/347))
+* TWBlue can upload images in Tweets and replies again. ([#240,](https://github.com/manuelcortez/TWBlue/issues/240))
+* Fixed the way we use to count characters in Twitter. The new methods in TWBlue take into account special characters and URLS as documented in Twitter. ([#199,](https://github.com/manuelcortez/TWBlue/issues/199) [#315](https://github.com/manuelcortez/TWBlue/issues/315))
+* Proxy support now works as expected.
+* Changed translation service from yandex.translate to Google Translator. ([#355,](https://github.com/manuelcortez/TWBlue/issues/355))
+* Improved method to load direct messages in the buffers. Now it should be faster due to less calls to Twitter API performed from the client.
+* And more. ([#352,](https://github.com/manuelcortez/TWBlue/issues/352))
 
 ## Changes in version 0.95
 

doc/documentation_importer.py

@@ -16,9 +16,9 @@ def prepare_documentation_in_file(fileSource, fileDest):
 		if "\n" == i:
 			newvar = "\"\","
 		elif "\n" == i[-1]:
-			newvar = "_(u\"\"\"%s\"\"\"),\n" % (i[:-1])
+			newvar = "\"\"\"%s\"\"\",\n" % (i[:-1])
 		else:
-			newvar = "_(u\"\"\"%s\"\"\"),\n" % (i)
+			newvar = "\"\"\"%s\"\"\",\n" % (i)
 		f2.write(newvar)
 	f1.close()
 	f2.write("]")

doc/generator.py

@@ -8,28 +8,27 @@ import shutil
 from codecs import open as _open
 from importlib import reload
 
-def change_language(name, language):
- global _
- os.environ["lang"] = language
- _ = gettext.install(name, os.path.join(paths.app_path(), "locales"))
+def get_translation_function(name, language):
+ if language == "en":
+  return gettext.NullTranslations()
+ translation_function = gettext.translation(name, os.path.join(paths.app_path(), "locales"), languages=[language])
+ return translation_function
 
 # the list of supported language codes of TW Blue
-languages = ["en", "es", "fr", "de", "it", "gl", "ja", "ru", "ro", "eu", "ca", "da"]
+languages = ["en", "es", "fr", "de", "it", "gl", "ja", "ru", "ro", "eu", "ca", "da", "sr"]
 
 def generate_document(language, document_type="documentation"):
  if document_type == "documentation":
   translation_file = "twblue-documentation"
-  change_language(translation_file, language)
-  reload(strings)
-  markdown_file = markdown.markdown("\n".join(strings.documentation[1:]), extensions=["markdown.extensions.toc"])
-  title = strings.documentation[0]
+  translation_function = get_translation_function(translation_file, language)
+  markdown_file = markdown.markdown("\n".join([translation_function.gettext(s[:-1]) if s != "\n" else s for s in strings.documentation[1:]]), extensions=["markdown.extensions.toc"])
+  title = translation_function.gettext(strings.documentation[0][:-1])
   filename = "manual.html"
  elif document_type == "changelog":
   translation_file = "twblue-changelog"
-  change_language(translation_file, language)
-  reload(changelog)
-  markdown_file = markdown.markdown("\n".join(changelog.documentation[1:]), extensions=["markdown.extensions.toc"])
-  title = changelog.documentation[0]
+  translation_function = get_translation_function(translation_file, language)
+  markdown_file = markdown.markdown("\n".join([translation_function.gettext(s[:-1]) if s != "\n" else s for s in changelog.documentation[1:]]), extensions=["markdown.extensions.toc"])
+  title = translation_function.gettext(changelog.documentation[0][:-1])
   filename = "changelog.html"
  first_html_block = """<!doctype html>
  <html lang="%s">
@@ -56,11 +55,13 @@ def create_documentation():
   shutil.copy(os.path.join("..", "license.txt"), os.path.join("documentation", "license.txt"))
  for i in languages:
   print("Creating documentation for: %s" % (i,))
+  try:
    generate_document(i)
    generate_document(i, "changelog")
+  except:
+   continue
  print("Done")
 
-change_language("twblue-documentation", "en")
 import strings
 import changelog
 create_documentation()

requirements.txt

@@ -1,4 +1,5 @@
-wxpython==4.0.3
+wxpython
+wheel
 six
 configobj
 markdown
@@ -15,8 +16,10 @@ futures
 winpaths
 PySocks
 win_inet_pton
-yandex.translate
-idna
+# Install the latest RC of this lib
+# see https://github.com/ssut/py-googletrans/issues/234
+googletrans==4.0.0-rc1
+idna<3,>=2.5
 chardet
 urllib3
 youtube-dl
@@ -25,8 +28,11 @@ pypiwin32
 certifi
 backports.functools_lru_cache
 cx_freeze
-git+https://github.com/manuelcortez/twython
-git+https://github.com/manuelcortez/libloader
-git+https://github.com/manuelcortez/platform_utils
-git+https://github.com/manuelcortez/accessible_output2
-git+https://github.com/jmdaweb/sound_lib
+tweepy
+twitter-text-parser
+pyenchant
+sqlitedict
+git+https://github.com/accessibleapps/libloader
+git+https://github.com/accessibleapps/platform_utils
+git+https://github.com/accessibleapps/accessible_output2
+git+https://github.com/accessibleapps/sound_lib

src/Conf.defaults

@@ -12,6 +12,7 @@ reverse_timelines = boolean(default=False)
 announce_stream_status = boolean(default=True)
 retweet_mode = string(default="ask")
 persist_size = integer(default=0)
+load_cache_in_memory=boolean(default=True)
 show_screen_names = boolean(default=False)
 buffer_order = list(default=list('home','mentions', 'dm', 'sent_dm', 'sent_tweets','favorites','followers','friends','blocks','muted','events'))
 

src/app-configuration.defaults

@@ -24,8 +24,8 @@ mention_all = boolean(default=False)
 no_streaming = boolean(default=False)
 
 [proxy]
-type = string(default="Direct connection")
+type = integer(default=0)
 server = string(default="")
-port = string(default="")
+port = integer(default=8080)
 user = string(default="")
 password = string(default="")

src/application.py

@@ -9,14 +9,14 @@ if snapshot == False:
     update_url = 'https://twblue.es/updates/stable.php'
     mirror_update_url = 'https://raw.githubusercontent.com/manuelcortez/TWBlue/next-gen/updates/stable.json'
 else:
- version = "1"
+    version = "6"
     update_url = 'https://twblue.es/updates/snapshot.php'
     mirror_update_url = 'https://raw.githubusercontent.com/manuelcortez/TWBlue/next-gen/updates/snapshots.json'
 authors = ["Manuel Cortéz", "José Manuel Delicado"]
 authorEmail = "manuel@manuelcortez.net"
-copyright = "Copyright (C) 2013-2018, Manuel cortéz."
+copyright = "Copyright (C) 2013-2021, Manuel cortéz."
 description = name+" is an app designed to use Twitter simply and efficiently while using minimal system resources. This app provides access to most Twitter  features."
-translators = ["Manuel Cortéz (English)", "Mohammed Al Shara, Hatoun Felemban (Arabic)", "Francisco Torres (Catalan)", "Manuel cortéz (Spanish)", "Sukil Etxenike Arizaleta (Basque)", "Jani Kinnunen (finnish)", "Rémy Ruiz (French)", "Juan Buño (Galician)", "Steffen Schultz (German)", "Zvonimir Stanečić (Croatian)", "Robert Osztolykan (Hungarian)", "Christian Leo Mameli (Italian)", "Riku (Japanese)", "Paweł Masarczyk (Polish)", "Odenilton Júnior Santos (Portuguese)", "Florian Ionașcu, Nicușor Untilă (Romanian)", "Natalia Hedlund, Valeria Kuznetsova (Russian)", "Aleksandar Đurić (Serbian)", "Burak Yüksek (Turkish)"]
+translators = ["Manuel Cortéz (English)", "Mohammed Al Shara, Hatoun Felemban (Arabic)", "Francisco Torres (Catalan)", "Manuel cortéz (Spanish)", "Sukil Etxenike Arizaleta (Basque)", "Jani Kinnunen (finnish)", "Oreonan (Français)", "Juan Buño (Galician)", "Steffen Schultz (German)", "Zvonimir Stanečić (Croatian)", "Robert Osztolykan (Hungarian)", "Christian Leo Mameli (Italian)", "Riku (Japanese)", "Paweł Masarczyk (Polish)", "Odenilton Júnior Santos (Portuguese)", "Florian Ionașcu, Nicușor Untilă (Romanian)", "Natalia Hedlund, Valeria Kuznetsova (Russian)", "Aleksandar Đurić (Serbian)", "Burak Yüksek (Turkish)"]
 url = u"https://twblue.es"
 report_bugs_url = "https://github.com/manuelcortez/twblue/issues"
 supported_languages = []

src/audio_services/services.py

@@ -1,9 +1,7 @@
 from __future__ import unicode_literals
 from audio_services import matches_url
-import youtube_utils
 import requests
-
-
+from . import youtube_utils
 
 @matches_url('https://audioboom.com')
 def convert_audioboom(url):

src/config.py

@@ -9,7 +9,7 @@ log = logging.getLogger("config")
 
 MAINFILE = "twblue.conf"
 MAINSPEC = "app-configuration.defaults"
-proxyTypes=[u"http", u"https", u"socks4", u"socks5"]
+proxyTypes = ["system", "http", "socks4", "socks4a", "socks5", "socks5h"]
 app = None
 keymap=None
 changed_keymap = False

src/controller/buffers/baseBuffers.py

@@ -59,11 +59,17 @@ class buffer(object):
         elif ev.GetKeyCode() == wx.WXK_F6: event = "volume_up"
         elif ev.GetKeyCode() == wx.WXK_DELETE and ev.ShiftDown(): event = "clear_list"
         elif ev.GetKeyCode() == wx.WXK_DELETE: event = "destroy_status"
+        # Raise a Special event when pressed Shift+F10 because Wx==4.1.x does not seems to trigger this by itself.
+        # See https://github.com/manuelcortez/TWBlue/issues/353
+        elif ev.GetKeyCode() == wx.WXK_F10 and ev.ShiftDown(): event = "show_menu"
         else:
             event = None
             ev.Skip()
         if event != None:
             try:
+                ### ToDo: Remove after WX fixes issue #353 in the widgets.
+                if event == "show_menu":
+                    return self.show_menu(widgetUtils.MENU, pos=self.buffer.list.list.GetPosition())
                 getattr(self, event)()
             except AttributeError:
                 pass

src/controller/buffers/twitterBuffers.py

@@ -1,7 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
-from builtins import str
-from builtins import range
 import time
 import platform
 if platform.system() == "Windows":
@@ -22,9 +19,10 @@ import languageHandler
 import logging
 from audio_services import youtube_utils
 from controller.buffers import baseBuffers
-from sessions.twitter import compose, utils
+from sessions.twitter import compose, utils, reduce
 from mysc.thread_utils import call_threaded
-from twython import TwythonError
+from tweepy.error import TweepError
+from tweepy.cursor import Cursor
 from pubsub import pub
 from sessions.twitter.long_tweets import twishort, tweets
 
@@ -86,6 +84,7 @@ class baseBufferController(baseBuffers.buffer):
         return _(u"Unknown buffer")
 
     def post_status(self, *args, **kwargs):
+        item = None
         title = _(u"Tweet")
         caption = _(u"Write the tweet here")
         tweet = messages.tweet(self.session, title, caption, "")
@@ -101,8 +100,8 @@ class baseBufferController(baseBuffers.buffer):
                     text = twishort.create_tweet(self.session.settings["twitter"]["user_key"], self.session.settings["twitter"]["user_secret"], text, 1)
             if not hasattr(tweet, "attachments") or len(tweet.attachments) == 0:
                 item = self.session.api_call(call_name="update_status", status=text, _sound="tweet_send.ogg", tweet_mode="extended")
-#   else:
-#    call_threaded(self.post_with_media, text=text, attachments=tweet.attachments, _sound="tweet_send.ogg")
+            else:
+                call_threaded(self.post_with_media, text=text, attachments=tweet.attachments)
             if item != None:
                 pub.sendMessage("sent-tweet", data=item, user=self.session.db["user_name"])
         if hasattr(tweet.message, "destroy"): tweet.message.destroy()
@@ -111,11 +110,12 @@ class baseBufferController(baseBuffers.buffer):
     def post_with_media(self, text, attachments):
         media_ids = []
         for i in attachments:
-   photo = open(i["file"], "rb")
-   img = self.session.twitter.upload_media(media=photo)
-   self.session.twitter.create_metadata(media_id=img["media_id"], alt_text=dict(text=i["description"]))
-   media_ids.append(img["media_id"])
-  self.session.twitter.update_status(status=text, media_ids=media_ids)
+            img = self.session.twitter.media_upload(i["file"])
+            self.session.twitter.create_media_metadata(media_id=img.media_id, alt_text=i["description"])
+            media_ids.append(img.media_id)
+        item = self.session.twitter.update_status(status=text, media_ids=media_ids)
+        if item != None:
+            pub.sendMessage("sent-tweet", data=item, user=self.session.db["user_name"])
 
     def get_formatted_message(self):
         if self.type == "dm" or self.name == "direct_messages":
@@ -129,31 +129,25 @@ class baseBufferController(baseBuffers.buffer):
     def get_full_tweet(self):
         tweet = self.get_right_tweet()
         tweetsList = []
-  tweet_id = tweet["id"]
+        tweet_id = tweet.id
         message = None
-  if "message" in tweet:
-   message = tweet["message"]
+        if hasattr(tweet, "message"):
+            message = tweet.message
         try:
-   tweet = self.session.twitter.show_status(id=tweet_id, include_ext_alt_text=True, tweet_mode="extended")
-   urls = utils.find_urls_in_text(tweet["full_text"])
-   for url in range(0, len(urls)):
-    try:  tweet["full_text"] = tweet["full_text"].replace(urls[url], tweet["entities"]["urls"][url]["expanded_url"])
-    except IndexError: pass
-  except TwythonError as e:
+            tweet = self.session.twitter.get_status(id=tweet_id, include_ext_alt_text=True, tweet_mode="extended")
+            tweet.full_text = utils.expand_urls(tweet.full_text, tweet.entities)
+        except TweepError as e:
             utils.twitter_error(e)
             return
         if message != None:
-   tweet["message"] = message
+            tweet.message = message
         l = tweets.is_long(tweet)
         while l != False:
             tweetsList.append(tweet)
             try:
-    tweet = self.session.twitter.show_status(id=l, include_ext_alt_text=True, tweet_mode="extended")
-    urls = utils.find_urls_in_text(tweet["full_text"])
-    for url in range(0, len(urls)):
-     try:  tweet["full_text"] = tweet["full_text"].replace(urls[url], tweet["entities"]["urls"][url]["expanded_url"])
-     except IndexError: pass
-   except TwythonError as e:
+                tweet = self.session.twitter.get_status(id=l, include_ext_alt_text=True, tweet_mode="extended")
+                tweet.full_text = utils.expand_urls(tweet.full_text, tweet.entities)
+            except TweepError as e:
                 utils.twitter_error(e)
                 return
             l = tweets.is_long(tweet)
@@ -168,18 +162,42 @@ class baseBufferController(baseBuffers.buffer):
             self.execution_time = current_time
             log.debug("Starting stream for buffer %s, account %s and type %s" % (self.name, self.account, self.type))
             log.debug("args: %s, kwargs: %s" % (self.args, self.kwargs))
-   if self.name == "direct_messages":
-    number_of_items = self.session.get_cursored_stream(self.name, self.function, *self.args, **self.kwargs)
-   else:
+            if self.name != "direct_messages":
                 val = self.session.call_paged(self.function, *self.args, **self.kwargs)
+            else:
+                # 50 results are allowed per API call, so let's assume max value can be 50.
+                # reference: https://developer.twitter.com/en/docs/twitter-api/v1/direct-messages/sending-and-receiving/api-reference/list-events
+                if self.session.settings["general"]["max_tweets_per_call"] > 50:
+                    count = 50
+                else:
+                    count = self.session.settings["general"]["max_tweets_per_call"]
+                # try to retrieve the cursor for the current buffer.
+                try:
+                    val = getattr(self.session.twitter, self.function)(return_cursors=True, count=count, *self.args, **self.kwargs)
+                    if type(val) == tuple:
+                        val, cursor = val
+                        if type(cursor) == tuple:
+                            cursor = cursor[1]
+                        cursors = self.session.db["cursors"]
+                        cursors[self.name] = cursor
+                        self.session.db["cursors"] = cursors
+                    results = [i for i in val]
+                    val = results
+                    val.reverse()
+                    log.debug("Retrieved %d items from the cursored search on function %s." %(len(val), self.function))
+                    user_ids = [item.message_create["sender_id"] for item in val]
+                    self.session.save_users(user_ids)
+                except TweepError as e:
+                    log.error("Error %s: %s" % (e.api_code, e.reason))
+                    return
             number_of_items = self.session.order_buffer(self.name, val)
             log.debug("Number of items retrieved: %d" % (number_of_items,))
             self.put_items_on_list(number_of_items)
             if hasattr(self, "finished_timeline") and self.finished_timeline == False:
                 if "-timeline" in self.name:
-     self.username = val[0]["user"]["screen_name"]
+                    self.username = val[0].user.screen_name
                 elif "-favorite" in self.name:
-     self.username = self.session.api_call("show_user", **self.kwargs)["screen_name"]
+                    self.username = self.session.api_call("get_user", **self.kwargs).screen_name
                 self.finished_timeline = True
             if number_of_items > 0 and self.name != "sent_tweets" and self.name != "sent_direct_messages" and self.sound != None and self.session.settings["sound"]["session_mute"] == False and self.name not in self.session.settings["other_buffers"]["muted_buffers"] and play_sound == True:
                 self.session.sound.play(self.sound)
@@ -202,25 +220,31 @@ class baseBufferController(baseBuffers.buffer):
     def get_more_items(self):
         elements = []
         if self.session.settings["general"]["reverse_timelines"] == False:
-   last_id = self.session.db[self.name][0]["id"]
+            last_id = self.session.db[self.name][0].id
         else:
-   last_id = self.session.db[self.name][-1]["id"]
+            last_id = self.session.db[self.name][-1].id
         try:
-   items = self.session.get_more_items(self.function, count=self.session.settings["general"]["max_tweets_per_call"], max_id=last_id, *self.args, **self.kwargs)
-  except TwythonError as e:
-   output.speak(e.message, True)
+            items = getattr(self.session.twitter, self.function)(max_id=last_id, count=self.session.settings["general"]["max_tweets_per_call"], *self.args, **self.kwargs)
+        except TweepError as e:
+            log.error("Error %s: %s" % (e.api_code, e.reason))
+            return
         if items == None:
             return
+        items_db = self.session.db[self.name]
+        self.session.add_users_from_results(items)
         for i in items:
-   if utils.is_allowed(i, self.session.settings, self.name) == True and utils.find_item(i["id"], self.session.db[self.name]) == None:
+            if utils.is_allowed(i, self.session.settings, self.name) == True and utils.find_item(i.id, self.session.db[self.name]) == None:
+                i = reduce.reduce_tweet(i)
                 i = self.session.check_quoted_status(i)
                 i = self.session.check_long_tweet(i)
                 elements.append(i)
                 if self.session.settings["general"]["reverse_timelines"] == False:
-     self.session.db[self.name].insert(0, i)
+                    items_db.insert(0, i)
                 else:
-     self.session.db[self.name].append(i)
+                    items_db.append(i)
+        self.session.db[self.name] = items_db
         selection = self.buffer.list.get_selected()
+        log.debug("Retrieved %d items from cursored search in function %s." % (len(elements), self.function))
         if self.session.settings["general"]["reverse_timelines"] == False:
             for i in elements:
                 tweet = self.compose_function(i, self.session.db, self.session.settings["general"]["relative_times"], self.session.settings["general"]["show_screen_names"], self.session)
@@ -229,8 +253,6 @@ class baseBufferController(baseBuffers.buffer):
             for i in items:
                 tweet = self.compose_function(i, self.session.db, self.session.settings["general"]["relative_times"], self.session.settings["general"]["show_screen_names"], self.session)
                 self.buffer.list.insert_item(False, *tweet)
-#   self.buffer.list.select_item(selection+elements)
-#  else:
             self.buffer.list.select_item(selection)
         output.speak(_(u"%s items retrieved") % (str(len(elements))), True)
 
@@ -269,17 +291,15 @@ class baseBufferController(baseBuffers.buffer):
 
     def remove_tweet(self, id):
         if type(self.session.db[self.name]) == dict: return
-  for i in range(0, len(self.session.db[self.name])):
-   if self.session.db[self.name][i]["id"] == id:
-    self.session.db[self.name].pop(i)
+        items = self.session.db[self.name]
+        for i in range(0, len(items)):
+            if items[i].id == id:
+                items.pop(i)
                 self.remove_item(i)
+        self.session.db[self.name] = items
 
     def put_items_on_list(self, number_of_items):
-  # Define the list we're going to use as cursored stuff are a bit different.
-  if self.name != "direct_messages" and self.name != "sent_direct_messages":
         list_to_use = self.session.db[self.name]
-  else:
-   list_to_use = self.session.db[self.name]["items"]
         if number_of_items == 0 and self.session.settings["general"]["persist_size"] == 0: return
         log.debug("The list contains %d items " % (self.buffer.list.get_count(),))
         log.debug("Putting %d items on the list" % (number_of_items,))
@@ -382,8 +402,8 @@ class baseBufferController(baseBuffers.buffer):
             self.show_menu(widgetUtils.MENU, pos=self.buffer.list.list.GetPosition())
 
     def get_tweet(self):
-  if "retweeted_status" in self.session.db[self.name][self.buffer.list.get_selected()]:
-   tweet = self.session.db[self.name][self.buffer.list.get_selected()]["retweeted_status"]
+        if hasattr(self.session.db[self.name][self.buffer.list.get_selected()], "retweeted_status"):
+            tweet = self.session.db[self.name][self.buffer.list.get_selected()].retweeted_status
         else:
             tweet = self.session.db[self.name][self.buffer.list.get_selected()]
         return tweet
@@ -395,11 +415,12 @@ class baseBufferController(baseBuffers.buffer):
     @_tweets_exist
     def reply(self, *args, **kwargs):
         tweet = self.get_right_tweet()
-  screen_name = tweet["user"]["screen_name"]
-  id = tweet["id"]
-  twishort_enabled = "twishort" in tweet
+        user = self.session.get_user(tweet.user)
+        screen_name = user.screen_name
+        id = tweet.id
+        twishort_enabled = hasattr(tweet, "twishort")
         users = utils.get_all_mentioned(tweet, self.session.db, field="screen_name")
-  ids = utils.get_all_mentioned(tweet, self.session.db, field="id_str")
+        ids = utils.get_all_mentioned(tweet, self.session.db, field="id")
         # Build the window title
         if len(users) < 1:
             title=_("Reply to {arg0}").format(arg0=screen_name)
@@ -432,7 +453,6 @@ class baseBufferController(baseBuffers.buffer):
             else:
                 params["call_name"] = "update_status_with_media"
                 params["media"] = message.file
-
             item = self.session.api_call(**params)
             if item != None:
                 pub.sendMessage("sent-tweet", data=item, user=self.session.db["user_name"])
@@ -443,44 +463,35 @@ class baseBufferController(baseBuffers.buffer):
     def send_message(self, *args, **kwargs):
         tweet = self.get_right_tweet()
         if self.type == "dm":
-   screen_name = self.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]
+            screen_name = self.session.get_user(tweet.message_create["sender_id"]).screen_name
             users = [screen_name]
         elif self.type == "people":
-   screen_name = tweet["screen_name"]
+            screen_name = tweet.screen_name
             users = [screen_name]
         else:
-   screen_name = tweet["user"]["screen_name"]
-   users = utils.get_all_users(tweet, self.session.db)
+            screen_name = self.session.get_user(tweet.user).screen_name
+            users = utils.get_all_users(tweet, self.session)
         dm = messages.dm(self.session, _(u"Direct message to %s") % (screen_name,), _(u"New direct message"), users)
         if dm.message.get_response() == widgetUtils.OK:
             screen_name = dm.message.get("cb")
             user = self.session.get_user_by_screen_name(screen_name)
-   event_data = {
-    'event': {
-        'type': 'message_create',
-        'message_create': {
-            'target': {
-                'recipient_id': user,
-            },
-            'message_data': {
-                'text': dm.message.get_text(),
-            }
-        }
-    }
-}
-   val = self.session.api_call(call_name="send_direct_message", **event_data)
+            recipient_id =  user
+            text = dm.message.get_text()
+            val = self.session.api_call(call_name="send_direct_message", recipient_id=recipient_id, text=text)
             if val != None:
+                sent_dms = self.session.db["sent_direct_messages"]
                 if self.session.settings["general"]["reverse_timelines"] == False:
-     self.session.db["sent_direct_messages"]["items"].append(val["event"])
+                    sent_dms.append(val)
                 else:
-     self.session.db["sent_direct_messages"]["items"].insert(0, val["event"])
-    pub.sendMessage("sent-dm", data=val["event"], user=self.session.db["user_name"])
+                    sent_dms.insert(0, val)
+                self.session.db["sent_direct_messages"] = sent_dms
+                pub.sendMessage("sent-dm", data=val, user=self.session.db["user_name"])
         if hasattr(dm.message, "destroy"): dm.message.destroy()
 
     @_tweets_exist
     def share_item(self, *args, **kwargs):
         tweet = self.get_right_tweet()
-  id = tweet["id"]
+        id = tweet.id
         if self.session.settings["general"]["retweet_mode"] == "ask":
             answer = commonMessageDialogs.retweet_question(self.buffer)
             if answer == widgetUtils.YES:
@@ -494,40 +505,40 @@ class baseBufferController(baseBuffers.buffer):
 
     def _retweet_with_comment(self, tweet, id, comment=''):
         # If quoting a retweet, let's quote the original tweet instead the retweet.
-  if "retweeted_status" in tweet:
-   tweet = tweet["retweeted_status"]
-  if "full_text" in tweet:
-   comments = tweet["full_text"]
+        if hasattr(tweet, "retweeted_status"):
+            tweet = tweet.retweeted_status
+        if hasattr(tweet, "full_text"):
+            comments = tweet.full_text
         else:
-   comments = tweet["text"]
-  retweet = messages.tweet(self.session, _(u"Quote"), _(u"Add your comment to the tweet"), u"“@%s: %s ”" % (tweet["user"]["screen_name"], comments), max=256, messageType="retweet")
+            comments = tweet.text
+        retweet = messages.tweet(self.session, _(u"Quote"), _(u"Add your comment to the tweet"), u"“@%s: %s ”" % (self.session.get_user(tweet.user).screen_name, comments), max=256, messageType="retweet")
         if comment != '':
             retweet.message.set_text(comment)
         if retweet.message.get_response() == widgetUtils.OK:
             text = retweet.message.get_text()
-   text = text+" https://twitter.com/{0}/status/{1}".format(tweet["user"]["screen_name"], id)
+            text = text+" https://twitter.com/{0}/status/{1}".format(self.session.get_user(tweet.user).screen_name, id)
             if retweet.image == None:
                 item = self.session.api_call(call_name="update_status", _sound="retweet_send.ogg", status=text, in_reply_to_status_id=id, tweet_mode="extended")
                 if item != None:
-     new_item = self.session.twitter.show_status(id=item["id"], include_ext_alt_text=True, tweet_mode="extended")
+                    new_item = self.session.twitter.get_status(id=item.id, include_ext_alt_text=True, tweet_mode="extended")
                     pub.sendMessage("sent-tweet", data=new_item, user=self.session.db["user_name"])
             else:
                 call_threaded(self.session.api_call, call_name="update_status", _sound="retweet_send.ogg", status=text, media=retweet.image)
         if hasattr(retweet.message, "destroy"): retweet.message.destroy()
 
     def _direct_retweet(self, id):
-  item = self.session.api_call(call_name="retweet", _sound="retweet_send.ogg", id=id, tweet_mode="extended")
+        item = self.session.api_call(call_name="retweet", _sound="retweet_send.ogg", id=id)
         if item != None:
             # Retweets are returned as non-extended tweets, so let's get the object as extended
             # just before sending the event message. See https://github.com/manuelcortez/TWBlue/issues/253
-   item = self.session.twitter.show_status(id=item["id"], include_ext_alt_text=True, tweet_mode="extended")
+            item = self.session.twitter.get_status(id=item.id, include_ext_alt_text=True, tweet_mode="extended")
             pub.sendMessage("sent-tweet", data=item, user=self.session.db["user_name"])
 
     def onFocus(self, *args, **kwargs):
         tweet = self.get_tweet()
         if platform.system() == "Windows" and self.session.settings["general"]["relative_times"] == True:
             # fix this:
-   original_date = arrow.get(self.session.db[self.name][self.buffer.list.get_selected()]["created_at"], "ddd MMM D H:m:s Z YYYY", locale="en")
+            original_date = arrow.get(self.session.db[self.name][self.buffer.list.get_selected()].created_at, locale="en")
             ts = original_date.humanize(locale=languageHandler.getLanguage())
             self.buffer.list.list.SetItem(self.buffer.list.get_selected(), 2, ts)
         if self.session.settings['sound']['indicate_audio'] and utils.is_audio(tweet):
@@ -542,7 +553,7 @@ class baseBufferController(baseBuffers.buffer):
             return sound.URLPlayer.stop_audio()
         tweet = self.get_tweet()
         if tweet == None: return
-  urls = utils.find_urls(tweet)
+        urls = utils.find_urls(tweet, twitter_media=True)
         if len(urls) == 1:
             url=urls[0]
         elif len(urls) > 1:
@@ -587,107 +598,122 @@ class baseBufferController(baseBuffers.buffer):
         if self.type == "events" or self.type == "people" or self.type == "empty" or self.type == "account": return
         answer = commonMessageDialogs.delete_tweet_dialog(None)
         if answer == widgetUtils.YES:
+            items = self.session.db[self.name]
             try:
                 if self.name == "direct_messages" or self.name == "sent_direct_messages":
-     self.session.twitter.destroy_direct_message(id=self.get_right_tweet()["id"])
-     self.session.db[self.name]["items"].pop(index)
+                    self.session.twitter.destroy_direct_message(id=self.get_right_tweet().id)
+                    items.pop(index)
                 else:
-     self.session.twitter.destroy_status(id=self.get_right_tweet()["id"])
-     self.session.db[self.name].pop(index)
+                    self.session.twitter.destroy_status(id=self.get_right_tweet().id)
+                    items.pop(index)
                 self.buffer.list.remove_item(index)
-   except TwythonError:
+            except TweepError:
                 self.session.sound.play("error.ogg")
+            self.session.db[self.name] = items
 
     @_tweets_exist
     def user_details(self):
         tweet = self.get_right_tweet()
         if self.type == "dm":
-   users = [self.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]]
+            users = [self.session.get_user(tweet.message_create["sender_id"]).screen_name]
         elif self.type == "people":
-   users = [tweet["screen_name"]]
+            users = [tweet.screen_name]
         else:
-   users = utils.get_all_users(tweet, self.session.db)
+            users = utils.get_all_users(tweet, self.session)
         dlg = dialogs.utils.selectUserDialog(title=_(u"User details"), users=users)
         if dlg.get_response() == widgetUtils.OK:
             user.profileController(session=self.session, user=dlg.get_user())
         if hasattr(dlg, "destroy"): dlg.destroy()
 
     def get_quoted_tweet(self, tweet):
-#  try:
-  quoted_tweet = self.session.twitter.show_status(id=tweet["id"])
-  urls = utils.find_urls_in_text(quoted_tweet["text"])
-  for url in range(0, len(urls)):
-   try:  quoted_tweet["text"] = quoted_tweet["text"].replace(urls[url], quoted_tweet["entities"]["urls"][url]["expanded_url"])
-   except IndexError: pass
-#  except TwythonError as e:
-#   utils.twitter_error(e)
-#   return
+        quoted_tweet = self.session.twitter.get_status(id=tweet.id)
+        quoted_tweet.text = utils.find_urls_in_text(quoted_tweet.text, quoted_tweet.entities)
         l = tweets.is_long(quoted_tweet)
         id = tweets.get_id(l)
-#  try:
-  original_tweet = self.session.twitter.show_status(id=id)
-  urls = utils.find_urls_in_text(original_tweet["text"])
-  for url in range(0, len(urls)):
-   try:  original_tweet["text"] = original_tweet["text"].replace(urls[url], original_tweet["entities"]["urls"][url]["expanded_url"])
-   except IndexError: pass
+        original_tweet = self.session.twitter.get_status(id=id)
+        original_tweet.text = utils.find_urls_in_text(original_tweet.text, original_tweet.entities)
         return compose.compose_quoted_tweet(quoted_tweet, original_tweet, self.session.db, self.session.settings["general"]["relative_times"])
 
     def open_in_browser(self, *args, **kwargs):
         tweet = self.get_tweet()
         output.speak(_(u"Opening item in web browser..."))
-  url = "https://twitter.com/{screen_name}/status/{tweet_id}".format(screen_name=tweet["user"]["screen_name"], tweet_id=tweet["id"])
+        url = "https://twitter.com/{screen_name}/status/{tweet_id}".format(screen_name=self.session.get_user(tweet.user).screen_name, tweet_id=tweet.id)
         webbrowser.open(url)
 
 class directMessagesController(baseBufferController):
 
     def get_more_items(self):
+        # 50 results are allowed per API call, so let's assume max value can be 50.
+        # reference: https://developer.twitter.com/en/docs/twitter-api/v1/direct-messages/sending-and-receiving/api-reference/list-events
+        if self.session.settings["general"]["max_tweets_per_call"] > 50:
+            count = 50
+        else:
+            count = self.session.settings["general"]["max_tweets_per_call"]
+        total = 0
+        # try to retrieve the cursor for the current buffer.
+        cursor = self.session.db["cursors"].get(self.name)
         try:
-   items = self.session.get_more_items(self.function, dm=True, name=self.name, count=self.session.settings["general"]["max_tweets_per_call"], cursor=self.session.db[self.name]["cursor"], *self.args, **self.kwargs)
-  except TwythonError as e:
-   output.speak(e.message, True)
+            items = getattr(self.session.twitter, self.function)(return_cursors=True, cursor=cursor, count=count, *self.args, **self.kwargs)
+            if type(items) == tuple:
+                items, cursor = items
+                if type(cursor) == tuple:
+                    cursor = cursor[1]
+                cursors = self.session.db["cursors"]
+                cursors[self.name] = cursor
+                self.session.db["cursors"] = cursors
+            results = [i for i in items]
+            items = results
+            log.debug("Retrieved %d items for cursored search in function %s" % (len(items), self.function))
+        except TweepError as e:
+            log.error("Error %s: %s" % (e.api_code, e.reason))
             return
         if items == None:
             return
         sent = []
+        received = []
+        sent_dms = self.session.db["sent_direct_messages"]
+        received_dms = self.session.db["direct_messages"]
         for i in items:
-   if i["message_create"]["sender_id"] == self.session.db["user_id"]:
+            if int(i.message_create["sender_id"]) == self.session.db["user_id"]:
                 if self.session.settings["general"]["reverse_timelines"] == False:
-     self.session.db["sent_direct_messages"]["items"].insert(0, i)
-    else:
-     self.session.db["sent_direct_messages"]["items"].append(i)
+                    sent_dms.insert(0, i)
                     sent.append(i)
                 else:
-    if self.session.settings["general"]["reverse_timelines"] == False:
-     self.session.db[self.name]["items"].insert(0, i)
+                    sent_dms.append(i)
+                    sent.insert(0, i)
             else:
-     self.session.db[self.name]["items"].append(i)
+                if self.session.settings["general"]["reverse_timelines"] == False:
+                    received_dms.insert(0, i)
+                    received.append(i)
+                else:
+                    received_dms.append(i)
+                    received.insert(0, i)
+            total = total+1
+        self.session.db["direct_messages"] = received_dms
+        self.session.db["sent_direct_messages"] = sent_dms
+        user_ids = [item.message_create["sender_id"] for item in items]
+        self.session.save_users(user_ids)
         pub.sendMessage("more-sent-dms", data=sent, account=self.session.db["user_name"])
         selected = self.buffer.list.get_selected()
         if self.session.settings["general"]["reverse_timelines"] == True:
-   for i in items:
-    if i["message_create"]["sender_id"] == self.session.db["user_id"]:
+            for i in received:
+                if int(i.message_create["sender_id"]) == self.session.db["user_id"]:
                     continue
                 tweet = self.compose_function(i, self.session.db, self.session.settings["general"]["relative_times"], self.session.settings["general"]["show_screen_names"], self.session)
                 self.buffer.list.insert_item(True, *tweet)
             self.buffer.list.select_item(selected)
         else:
-   for i in items:
-    if i["message_create"]["sender_id"] == self.session.db["user_id"]:
+            for i in received:
+                if int(i.message_create["sender_id"]) == self.session.db["user_id"]:
                     continue
                 tweet = self.compose_function(i, self.session.db, self.session.settings["general"]["relative_times"], self.session.settings["general"]["show_screen_names"], self.session)
                 self.buffer.list.insert_item(True, *tweet)
-  output.speak(_(u"%s items retrieved") % (len(items)), True)
-
- def get_tweet(self):
-  tweet = self.session.db[self.name]["items"][self.buffer.list.get_selected()]
-  return tweet
-
- get_right_tweet = get_tweet
+        output.speak(_(u"%s items retrieved") % (total), True)
 
     @_tweets_exist
     def reply(self, *args, **kwargs):
         tweet = self.get_right_tweet()
-  screen_name = self.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]
+        screen_name = self.session.get_user(tweet.message_create["sender_id"]).screen_name
         message = messages.reply(self.session, _(u"Mention"), _(u"Mention to %s") % (screen_name,), "@%s " % (screen_name,), [screen_name,])
         if message.message.get_response() == widgetUtils.OK:
             if config.app["app-settings"]["remember_mention_and_longtweet"]:
@@ -705,7 +731,7 @@ class directMessagesController(baseBufferController):
         tweet = self.get_tweet()
         if platform.system() == "Windows" and self.session.settings["general"]["relative_times"] == True:
             # fix this:
-   original_date = arrow.get(int(tweet["created_timestamp"][:-3]))
+            original_date = arrow.get(int(tweet.created_timestamp))
             ts = original_date.humanize(locale=languageHandler.getLanguage())
             self.buffer.list.list.SetItem(self.buffer.list.get_selected(), 2, ts)
         if self.session.settings['sound']['indicate_audio'] and utils.is_audio(tweet):
@@ -716,15 +742,15 @@ class directMessagesController(baseBufferController):
     def clear_list(self):
         dlg = commonMessageDialogs.clear_list()
         if dlg == widgetUtils.YES:
-   self.session.db[self.name]["items"] = []
+            self.session.db[self.name] = []
             self.buffer.list.clear()
 
     def auto_read(self, number_of_items):
         if number_of_items == 1 and self.name in self.session.settings["other_buffers"]["autoread_buffers"] and self.name not in self.session.settings["other_buffers"]["muted_buffers"] and self.session.settings["sound"]["session_mute"] == False:
             if self.session.settings["general"]["reverse_timelines"] == False:
-    tweet = self.session.db[self.name]["items"][-1]
+                tweet = self.session.db[self.name][-1]
             else:
-    tweet = self.session.db[self.name]["items"][0]
+                tweet = self.session.db[self.name][0]
             output.speak(_(u"New direct message"))
             output.speak(" ".join(self.compose_function(tweet, self.session.db, self.session.settings["general"]["relative_times"], self.session.settings["general"]["show_screen_names"], self.session)))
         elif number_of_items > 1 and self.name in self.session.settings["other_buffers"]["autoread_buffers"] and self.name not in self.session.settings["other_buffers"]["muted_buffers"] and self.session.settings["sound"]["session_mute"] == False:
@@ -738,7 +764,7 @@ class sentDirectMessagesController(directMessagesController):
     def __init__(self, *args, **kwargs):
         super(sentDirectMessagesController, self).__init__(*args, **kwargs)
         if ("sent_direct_messages" in self.session.db) == False:
-   self.session.db["sent_direct_messages"] = {"items": []}
+            self.session.db["sent_direct_messages"] = []
 
     def get_more_items(self):
         output.speak(_(u"Getting more items cannot be done in this buffer. Use the direct messages buffer instead."))
@@ -750,11 +776,11 @@ class sentDirectMessagesController(directMessagesController):
         if self.session.settings["general"]["reverse_timelines"] == True:
             for i in items:
                 tweet = self.compose_function(i, self.session.db, self.session.settings["general"]["relative_times"], self.session.settings["general"]["show_screen_names"], self.session)
-    self.buffer.list.insert_item(True, *tweet)
+                self.buffer.list.insert_item(False, *tweet)
         else:
             for i in items:
                 tweet = self.compose_function(i, self.session.db, self.session.settings["general"]["relative_times"], self.session.settings["general"]["show_screen_names"], self.session)
-    self.buffer.list.insert_item(True, *tweet)
+                self.buffer.list.insert_item(False, *tweet)
 
 class listBufferController(baseBufferController):
     def __init__(self, parent, function, name, sessionObject, account, sound=None, bufferType=None, list_id=None, *args, **kwargs):
@@ -768,13 +794,9 @@ class listBufferController(baseBufferController):
         super(listBufferController, self).start_stream(mandatory, play_sound, avoid_autoreading)
 
     def get_user_ids(self):
-  next_cursor = -1
-  while(next_cursor):
-   users = self.session.twitter.get_list_members(list_id=self.list_id, cursor=next_cursor, include_entities=False, skip_status=True)
-   for i in users['users']:
-    if i["id"] not in self.users:
-     self.users.append(i["id"])
-    next_cursor = users["next_cursor"]
+        for i in Cursor(self.session.twitter.list_members, list_id=self.list_id, include_entities=False, skip_status=True, count=5000).items():
+            if i.id not in self.users:
+                self.users.append(i.id)
 
     def remove_buffer(self, force=False):
         if force == False:
@@ -854,7 +876,7 @@ class peopleBufferController(baseBufferController):
     @_tweets_exist
     def reply(self, *args, **kwargs):
         tweet = self.get_right_tweet()
-  screen_name = tweet["screen_name"]
+        screen_name = tweet.screen_name
         message = messages.reply(self.session, _(u"Mention"), _(u"Mention to %s") % (screen_name,), "@%s " % (screen_name,), [screen_name,])
         if message.message.get_response() == widgetUtils.OK:
             if config.app["app-settings"]["remember_mention_and_longtweet"]:
@@ -875,31 +897,61 @@ class peopleBufferController(baseBufferController):
             self.execution_time = current_time
             log.debug("Starting stream for %s buffer, %s account" % (self.name, self.account,))
             log.debug("args: %s, kwargs: %s" % (self.args, self.kwargs))
-   val = self.session.get_cursored_stream(self.name, self.function, *self.args, **self.kwargs)
-   self.put_items_on_list(val)
+            try:
+                val = getattr(self.session.twitter, self.function)(return_cursors=True, count=self.session.settings["general"]["max_tweets_per_call"], *self.args, **self.kwargs)
+                if type(val) == tuple:
+                    val, cursor = val
+                    if type(cursor) == tuple:
+                        cursor = cursor[1]
+                    cursors = self.session.db["cursors"]
+                    cursors[self.name] = cursor
+                    self.session.db["cursors"] = cursors
+                results = [i for i in val]
+                val = results
+                val.reverse()
+                log.debug("Retrieved %d items from cursored search in function %s" % (len(val), self.function))
+            except TweepError as e:
+                log.error("Error %s: %s" % (e.api_code, e.reason))
+                return
+            number_of_items = self.session.order_people(self.name, val)
+            log.debug("Number of items retrieved: %d" % (number_of_items,))
+            self.put_items_on_list(number_of_items)
             if hasattr(self, "finished_timeline") and self.finished_timeline == False:
-    self.username = self.session.api_call("show_user", **self.kwargs)["screen_name"]
+                self.username = self.session.api_call("get_user", **self.kwargs).screen_name
                 self.finished_timeline = True
-   if val > 0 and self.sound != None and self.session.settings["sound"]["session_mute"] == False and self.name not in self.session.settings["other_buffers"]["muted_buffers"] and play_sound == True:
+            if number_of_items > 0 and self.sound != None and self.session.settings["sound"]["session_mute"] == False and self.name not in self.session.settings["other_buffers"]["muted_buffers"] and play_sound == True:
                 self.session.sound.play(self.sound)
             # Autoread settings
-   if avoid_autoreading == False and mandatory == True and val > 0 and self.name in self.session.settings["other_buffers"]["autoread_buffers"]:
-    self.auto_read(val)
-   return val
+            if avoid_autoreading == False and mandatory == True and number_of_items > 0 and self.name in self.session.settings["other_buffers"]["autoread_buffers"]:
+                self.auto_read(number_of_items)
+            return number_of_items
 
     def get_more_items(self):
         try:
-   items = self.session.get_more_items(self.function, users=True, name=self.name, count=self.session.settings["general"]["max_tweets_per_call"], cursor=self.session.db[self.name]["cursor"], *self.args, **self.kwargs)
-  except TwythonError as e:
-   output.speak(e.message, True)
+            cursor = self.session.db["cursors"].get(self.name)
+            items = getattr(self.session.twitter, self.function)(return_cursors=True, users=True, cursor=cursor, count=self.session.settings["general"]["max_tweets_per_call"], *self.args, **self.kwargs)
+            if type(items) == tuple:
+                items, cursor = items
+                if type(cursor) == tuple:
+                    cursor = cursor[1]
+                cursors = self.session.db["cursors"]
+                cursors[self.name] = cursor
+                self.session.db["cursors"] = cursors
+            results = [i for i in items]
+            items = results
+            log.debug("Retrieved %d items from cursored search in function %s" % (len(items), self.function))
+        except TweepError as e:
+            log.error("Error %s: %s" % (e.api_code, e.reason))
             return
         if items == None:
             return
+        items_db = self.session.db[self.name]
         for i in items:
             if self.session.settings["general"]["reverse_timelines"] == False:
-    self.session.db[self.name]["items"].insert(0, i)
+                items_db.insert(0, i)
             else:
-    self.session.db[self.name]["items"].append(i)
+                items_db.append(i)
+        self.session.db[self.name] = items_db
         selected = self.buffer.list.get_selected()
         if self.session.settings["general"]["reverse_timelines"] == True:
             for i in items:
@@ -916,18 +968,18 @@ class peopleBufferController(baseBufferController):
         log.debug("The list contains %d items" % (self.buffer.list.get_count(),))
 #  log.debug("Putting %d items on the list..." % (number_of_items,))
         if self.buffer.list.get_count() == 0:
-   for i in self.session.db[self.name]["items"]:
+            for i in self.session.db[self.name]:
                 tweet = self.compose_function(i, self.session.db, self.session.settings["general"]["relative_times"], self.session)
                 self.buffer.list.insert_item(False, *tweet)
             self.buffer.set_position(self.session.settings["general"]["reverse_timelines"])
 #   self.buffer.set_list_position()
         elif self.buffer.list.get_count() > 0:
             if self.session.settings["general"]["reverse_timelines"] == False:
-    for i in self.session.db[self.name]["items"][len(self.session.db[self.name]["items"])-number_of_items:]:
+                for i in self.session.db[self.name][len(self.session.db[self.name])-number_of_items:]:
                     tweet = self.compose_function(i, self.session.db)
                     self.buffer.list.insert_item(False, *tweet)
             else:
-    items = self.session.db[self.name]["items"][0:number_of_items]
+                items = self.session.db[self.name][0:number_of_items]
                 items.reverse()
                 for i in items:
                     tweet = self.compose_function(i, self.session.db)
@@ -935,7 +987,7 @@ class peopleBufferController(baseBufferController):
         log.debug("now the list contains %d items" % (self.buffer.list.get_count(),))
 
     def get_right_tweet(self):
-  tweet = self.session.db[self.name]["items"][self.buffer.list.get_selected()]
+        tweet = self.session.db[self.name][self.buffer.list.get_selected()]
         return tweet
 
     def add_new_item(self, item):
@@ -950,12 +1002,12 @@ class peopleBufferController(baseBufferController):
     def clear_list(self):
         dlg = commonMessageDialogs.clear_list()
         if dlg == widgetUtils.YES:
-   self.session.db[self.name]["items"] = []
-   self.session.db[self.name]["cursor"] = -1
+            self.session.db[self.name] = []
+            self.session.db["cursors"][self.name] = -1
             self.buffer.list.clear()
 
     def interact(self):
-  user.profileController(self.session, user=self.get_right_tweet()["screen_name"])
+        user.profileController(self.session, user=self.get_right_tweet().screen_name)
 
     def show_menu(self, ev, pos=0, *args, **kwargs):
         menu = menus.peoplePanelMenu()
@@ -978,9 +1030,9 @@ class peopleBufferController(baseBufferController):
     def auto_read(self, number_of_items):
         if number_of_items == 1 and self.name in self.session.settings["other_buffers"]["autoread_buffers"] and self.name not in self.session.settings["other_buffers"]["muted_buffers"] and self.session.settings["sound"]["session_mute"] == False:
             if self.session.settings["general"]["reverse_timelines"] == False:
-    tweet = self.session.db[self.name]["items"][-1]
+                tweet = self.session.db[self.name][-1]
             else:
-    tweet = self.session.db[self.name["items"]][0]
+                tweet = self.session.db[self.name][0]
             output.speak(" ".join(self.compose_function(tweet, self.session.db, self.session.settings["general"]["relative_times"], self.session.settings["general"]["show_screen_names"], self.session)))
         elif number_of_items > 1 and self.name in self.session.settings["other_buffers"]["autoread_buffers"] and self.name not in self.session.settings["other_buffers"]["muted_buffers"] and self.session.settings["sound"]["session_mute"] == False:
             output.speak(_(u"{0} new followers.").format(number_of_items))
@@ -988,30 +1040,10 @@ class peopleBufferController(baseBufferController):
     def open_in_browser(self, *args, **kwargs):
         tweet = self.get_tweet()
         output.speak(_(u"Opening item in web browser..."))
-  url = "https://twitter.com/{screen_name}".format(screen_name=tweet["screen_name"])
+        url = "https://twitter.com/{screen_name}".format(screen_name=tweet.screen_name)
         webbrowser.open(url)
 
 class searchBufferController(baseBufferController):
- def start_stream(self, mandatory=False, play_sound=True, avoid_autoreading=False):
-  # starts stream every 3 minutes.
-  current_time = time.time()
-  if self.execution_time == 0 or current_time-self.execution_time >= 180 or mandatory==True:
-   self.execution_time = current_time
-   log.debug("Starting stream for %s buffer, %s account and %s type" % (self.name, self.account, self.type))
-   log.debug("args: %s, kwargs: %s" % (self.args, self.kwargs))
-   log.debug("Function: %s" % (self.function,))
-#  try:
-   val = self.session.search(self.name, count=self.session.settings["general"]["max_tweets_per_call"], *self.args, **self.kwargs)
-#  except:
-#   return None
-   num = self.session.order_buffer(self.name, val)
-   self.put_items_on_list(num)
-   if num > 0 and self.sound != None and self.session.settings["sound"]["session_mute"] == False and self.name not in self.session.settings["other_buffers"]["muted_buffers"] and play_sound == True:
-    self.session.sound.play(self.sound)
-   # Autoread settings
-   if avoid_autoreading == False and mandatory == True and num > 0 and self.name in self.session.settings["other_buffers"]["autoread_buffers"]:
-    self.auto_read(num)
-   return num
 
     def remove_buffer(self, force=False):
         if force == False:
@@ -1028,105 +1060,22 @@ class searchBufferController(baseBufferController):
         elif dlg == widgetUtils.NO:
             return False
 
- def get_more_items(self):
-  elements = []
-  if self.session.settings["general"]["reverse_timelines"] == False:
-   last_id = self.session.db[self.name][0]["id"]
-  else:
-   last_id = self.session.db[self.name][-1]["id"]
-  try:
-   items = self.session.search(self.name, count=self.session.settings["general"]["max_tweets_per_call"], max_id=last_id, *self.args, **self.kwargs)
-  except TwythonError as e:
-   output.speak(e.message, True)
-  if items == None:
-   return
-  for i in items:
-   if utils.is_allowed(i, self.session.settings, self.name) == True and utils.find_item(i["id"], self.session.db[self.name]) == None:
-    i = self.session.check_quoted_status(i)
-    i = self.session.check_long_tweet(i)
-    elements.append(i)
-    if self.session.settings["general"]["reverse_timelines"] == False:
-     self.session.db[self.name].insert(0, i)
-    else:
-     self.session.db[self.name].append(i)
-  selection = self.buffer.list.get_selected()
-  if self.session.settings["general"]["reverse_timelines"] == False:
-   for i in elements:
-    tweet = self.compose_function(i, self.session.db, self.session.settings["general"]["relative_times"], self.session.settings["general"]["show_screen_names"], self.session)
-    self.buffer.list.insert_item(True, *tweet)
-  else:
-   for i in items:
-    tweet = self.compose_function(i, self.session.db, self.session.settings["general"]["relative_times"], self.session.settings["general"]["show_screen_names"], self.session)
-    self.buffer.list.insert_item(False, *tweet)
-#   self.buffer.list.select_item(selection+elements)
-#  else:
-   self.buffer.list.select_item(selection)
-  output.speak(_(u"%s items retrieved") % (str(len(elements))), True)
-
 class searchPeopleBufferController(peopleBufferController):
-
+    """ This is identical to a normal peopleBufferController, except that uses the page parameter instead of a cursor."""
     def __init__(self, parent, function, name, sessionObject, account, bufferType="peoplePanel", *args, **kwargs):
         super(searchPeopleBufferController, self).__init__(parent, function, name, sessionObject, account, bufferType="peoplePanel", *args, **kwargs)
-  log.debug("Initializing buffer %s, account %s" % (name, account,))
-#  self.compose_function = compose.compose_followers_list
-  log.debug("Compose_function: %s" % (self.compose_function,))
-  self.args = args
-  self.kwargs = kwargs
-  self.function = function
         if ("page" in self.kwargs) == False:
-   self.kwargs["page"] = 1
-
- def start_stream(self, mandatory=False, play_sound=True, avoid_autoreading=True):
-  # starts stream every 3 minutes.
-  current_time = time.time()
-  if self.execution_time == 0 or current_time-self.execution_time >= 180 or mandatory==True:
-   self.execution_time = current_time
-   log.debug("starting stream for %s buffer, %s account and %s type" % (self.name, self.account, self.type))
-   log.debug("args: %s, kwargs: %s" % (self.args, self.kwargs))
-   log.debug("Function: %s" % (self.function,))
-#  try:
-   val = self.session.call_paged(self.function, *self.args, **self.kwargs)
-#  except:
-#   return
-   number_of_items = self.session.order_cursored_buffer(self.name, val)
-   log.debug("Number of items retrieved: %d" % (number_of_items,))
-   self.put_items_on_list(number_of_items)
-   if number_of_items > 0 and self.sound != None and self.session.settings["sound"]["session_mute"] == False and self.name not in self.session.settings["other_buffers"]["muted_buffers"] and play_sound == True:
-    self.session.sound.play(self.sound)
-   # Autoread settings
-   if avoid_autoreading == False and mandatory == True and number_of_items > 0 and self.name in self.session.settings["other_buffers"]["autoread_buffers"]:
-    self.auto_read(number_of_items)
-   return number_of_items
+            self.page = 1
+        else:
+            self.page = self.kwargs.pop("page")
 
     def get_more_items(self, *args, **kwargs):
-  self.kwargs["page"] += 1
-  try:
-   items = self.session.get_more_items(self.function, users=True, name=self.name, count=self.session.settings["general"]["max_tweets_per_call"],  *self.args, **self.kwargs)
-  except TwythonError as e:
-   output.speak(e.message, True)
-   return
-  if items == None:
-   return
-  for i in items:
-   if self.session.settings["general"]["reverse_timelines"] == False:
-    self.session.db[self.name]["items"].insert(0, i)
-   else:
-    self.session.db[self.name]["items"].append(i)
-  selected = self.buffer.list.get_selected()
-#  self.put_items_on_list(len(items))
-  if self.session.settings["general"]["reverse_timelines"] == True:
-   for i in items:
-    tweet = self.compose_function(i, self.session.db, self.session.settings["general"]["relative_times"], self.session)
-    self.buffer.list.insert_item(True, *tweet)
-   self.buffer.list.select_item(selected)
-  else:
-   for i in items:
-    tweet = self.compose_function(i, self.session.db, self.session.settings["general"]["relative_times"], self.session)
-    self.buffer.list.insert_item(True, *tweet)
-#   self.buffer.list.select_item(selection)
-#  else:
-#   self.buffer.list.select_item(selection-elements)
-  output.speak(_(u"%s items retrieved") % (len(items)), True)
+        # Add 1 to the page parameter, put it in kwargs and calls to get_more_items in the parent buffer.
+        self.page = self.page +1
+        self.kwargs["page"] = self.page
+        super(searchPeopleBufferController, self).get_more_items(*args, **kwargs)
+        # remove the parameter again to make sure start_stream won't fetch items for this page indefinitely.
+        self.kwargs.pop("page")
 
     def remove_buffer(self, force=False):
         if force == False:
@@ -1168,9 +1117,9 @@ class trendsBufferController(baseBuffers.buffer):
         if self.execution_time == 0 or current_time-self.execution_time >= 180 or mandatory == True:
             self.execution_time = current_time
             try:
-    data = self.session.call_paged("get_place_trends", id=self.trendsFor)
-   except:
-    return
+                data = self.session.twitter.trends_place(id=self.trendsFor)
+            except TweepError as err:
+                log.error("Error %s: %s" % (err.api_code, err.reason))
             if not hasattr(self, "name_"):
                 self.name_ = data[0]["locations"][0]["name"]
             self.trends = data[0]["trends"]
@@ -1279,23 +1228,25 @@ class conversationBufferController(searchBufferController):
                 self.statuses = []
                 self.ids = []
                 self.statuses.append(self.tweet)
-    self.ids.append(self.tweet["id"])
+                self.ids.append(self.tweet.id)
                 tweet = self.tweet
-    while tweet["in_reply_to_status_id"] != None:
+                if not hasattr(tweet, "in_reply_to_status_id"):
+                    tweet.in_reply_to_status_id = None
+                while tweet.in_reply_to_status_id != None:
                     try:
-      tweet = self.session.twitter.show_status(id=tweet["in_reply_to_status_id"], tweet_mode="extended")
-     except TwythonError as err:
+                        tweet = self.session.twitter.get_status(id=tweet.in_reply_to_status_id, tweet_mode="extended")
+                    except TweepError as err:
                         break
                     self.statuses.insert(0, tweet)
-     self.ids.append(tweet["id"])
-    if tweet["in_reply_to_status_id"] == None:
-     self.kwargs["since_id"] = tweet["id"]
-     self.ids.append(tweet["id"])
+                    self.ids.append(tweet.id)
+                if tweet.in_reply_to_status_id == None:
+                    self.kwargs["since_id"] = tweet.id
+                    self.ids.append(tweet.id)
             val2 = self.session.search(self.name, tweet_mode="extended", *self.args, **self.kwargs)
             for i in val2:
-    if i["in_reply_to_status_id"] in self.ids:
+                if i.in_reply_to_status_id in self.ids:
                     self.statuses.append(i)
-     self.ids.append(i["id"])
+                    self.ids.append(i.id)
                     tweet = i
             number_of_items = self.session.order_buffer(self.name, self.statuses)
             log.debug("Number of items retrieved: %d" % (number_of_items,))

src/controller/listsController.py

@@ -1,13 +1,14 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
-from builtins import object
 import widgetUtils
 import output
+import logging
 from wxUI.dialogs import lists
-from twython import TwythonError
+from tweepy.error import TweepError
 from sessions.twitter import compose, utils
 from pubsub import pub
 
+log = logging.getLogger("controller.listsController")
+
 class listsController(object):
     def __init__(self, session, user=None):
         super(listsController, self).__init__()
@@ -31,7 +32,7 @@ class listsController(object):
         return [compose.compose_list(item) for item in self.session.db["lists"]]
 
     def get_user_lists(self, user):
-  self.lists = self.session.twitter.show_lists(reverse=True, screen_name=user)
+        self.lists = self.session.twitter.lists_all(reverse=True, screen_name=user)
         return [compose.compose_list(item) for item in self.lists]
 
     def create_list(self, *args, **kwargs):
@@ -48,8 +49,9 @@ class listsController(object):
                 new_list = self.session.twitter.create_list(name=name, description=description, mode=mode)
                 self.session.db["lists"].append(new_list)
                 self.dialog.lista.insert_item(False, *compose.compose_list(new_list))
-   except TwythonError as e:
-    output.speak("error %s: %s" % (e.status_code, e.msg))
+            except TweepError as e:
+                output.speak("error %s: %s" % (e.api_code, e.reason))
+                log.exception("error %s: %s" % (e.api_code, e.reason))
         dialog.destroy()
 
     def edit_list(self, *args, **kwargs):
@@ -65,44 +67,44 @@ class listsController(object):
             else:
                 mode = "private"
             try:
-    self.session.twitter.update_list(list_id=list["id"], name=name, description=description, mode=mode)
+                self.session.twitter.update_list(list_id=list.id, name=name, description=description, mode=mode)
                 self.session.get_lists()
                 self.dialog.populate_list(self.get_all_lists(), True)
-   except TwythonError as e:
-    output.speak("error %s: %s" % (e.error_code, e.msg))
+            except TweepError as e:
+                output.speak("error %s: %s" % (e.api_code, e.reason))
         dialog.destroy()
 
     def remove_list(self, *args, **kwargs):
         if self.dialog.lista.get_count() == 0: return
-  list = self.session.db["lists"][self.dialog.get_item()]["id"]
+        list = self.session.db["lists"][self.dialog.get_item()].id
         if lists.remove_list() == widgetUtils.YES:
             try:
-    self.session.twitter.delete_list(list_id=list)
+                self.session.twitter.destroy_list(list_id=list)
                 self.session.db["lists"].pop(self.dialog.get_item())
                 self.dialog.lista.remove_item(self.dialog.get_item())
-   except TwythonError as e:
-    output.speak("error %s: %s" % (e.error_code, e.msg))
+            except TweepError as e:
+                output.speak("error %s: %s" % (e.api_code, e.reason))
 
     def open_list_as_buffer(self, *args, **kwargs):
         if self.dialog.lista.get_count() == 0: return
         list = self.session.db["lists"][self.dialog.get_item()]
-  pub.sendMessage("create-new-buffer", buffer="list", account=self.session.db["user_name"], create=list["name"])
+        pub.sendMessage("create-new-buffer", buffer="list", account=self.session.db["user_name"], create=list.name)
 
     def subscribe(self, *args, **kwargs):
         if self.dialog.lista.get_count() == 0: return
-  list_id = self.lists[self.dialog.get_item()]["id"]
+        list_id = self.lists[self.dialog.get_item()].id
         try:
-   list = self.session.twitter.subscribe_to_list(list_id=list_id)
-   item = utils.find_item(list["id"], self.session.db["lists"])
+            list = self.session.twitter.subscribe_list(list_id=list_id)
+            item = utils.find_item(list.id, self.session.db["lists"])
             self.session.db["lists"].append(list)
-  except TwythonError as e:
-   output.speak("error %s: %s" % (e.status_code, e.msg))
+        except TweepError as e:
+            output.speak("error %s: %s" % (e.api_code, e.reason))
 
     def unsubscribe(self, *args, **kwargs):
         if self.dialog.lista.get_count() == 0: return
-  list_id = self.lists[self.dialog.get_item()]["id"]
+        list_id = self.lists[self.dialog.get_item()].id
         try:
-   list = self.session.twitter.unsubscribe_from_list(list_id=list_id)
+            list = self.session.twitter.unsubscribe_list(list_id=list_id)
             self.session.db["lists"].remove(list)
-  except TwythonError as e:
-   output.speak("error %s: %s" % (e.status_code, e.msg))
+        except TweepError as e:
+            output.speak("error %s: %s" % (e.api_code, e.reason))

src/controller/mainController.py

@@ -1,8 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
-from builtins import str
-from builtins import range
-from builtins import object
 import platform
 system = platform.system()
 import application
@@ -34,7 +30,7 @@ from sessions.twitter  import session as session_
 from pubsub import pub
 import sound
 import output
-from twython import TwythonError, TwythonAuthError
+from tweepy.error import TweepError
 from mysc.thread_utils import call_threaded
 from mysc.repeating_timer import RepeatingTimer
 from mysc import restart
@@ -257,9 +253,9 @@ class Controller(object):
 
         # Connection checker executed each minute.
         self.checker_function = RepeatingTimer(60, self.check_connection)
-  self.checker_function.start()
-  self.save_db = RepeatingTimer(300, self.save_data_in_db)
-  self.save_db.start()
+#  self.checker_function.start()
+#        self.save_db = RepeatingTimer(300, self.save_data_in_db)
+#        self.save_db.start()
         log.debug("Setting updates to buffers every %d seconds..." % (60*config.app["app-settings"]["update_period"],))
         self.update_buffers_function = RepeatingTimer(60*config.app["app-settings"]["update_period"], self.update_buffers)
         self.update_buffers_function.start()
@@ -304,15 +300,15 @@ class Controller(object):
             self.view.add_buffer(account.buffer , name=session.db["user_name"])
         for i in session.settings['general']['buffer_order']:
             if i == 'home':
-    home = twitterBuffers.baseBufferController(self.view.nb, "get_home_timeline", "home_timeline", session, session.db["user_name"], sound="tweet_received.ogg", tweet_mode="extended")
+                home = twitterBuffers.baseBufferController(self.view.nb, "home_timeline", "home_timeline", session, session.db["user_name"], sound="tweet_received.ogg", tweet_mode="extended")
                 self.buffers.append(home)
                 self.view.insert_buffer(home.buffer, name=_(u"Home"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
             elif i == 'mentions':
-    mentions = twitterBuffers.baseBufferController(self.view.nb, "get_mentions_timeline", "mentions", session, session.db["user_name"], sound="mention_received.ogg", tweet_mode="extended")
+                mentions = twitterBuffers.baseBufferController(self.view.nb, "mentions_timeline", "mentions", session, session.db["user_name"], sound="mention_received.ogg", tweet_mode="extended")
                 self.buffers.append(mentions)
                 self.view.insert_buffer(mentions.buffer, name=_(u"Mentions"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
             elif i == 'dm':
-    dm = twitterBuffers.directMessagesController(self.view.nb, "get_direct_messages", "direct_messages", session, session.db["user_name"], bufferType="dmPanel", compose_func="compose_direct_message", sound="dm_received.ogg", full_text=True, items="events")
+                dm = twitterBuffers.directMessagesController(self.view.nb, "list_direct_messages", "direct_messages", session, session.db["user_name"], bufferType="dmPanel", compose_func="compose_direct_message", sound="dm_received.ogg")
                 self.buffers.append(dm)
                 self.view.insert_buffer(dm.buffer, name=_(u"Direct messages"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
             elif i == 'sent_dm':
@@ -320,62 +316,62 @@ class Controller(object):
                 self.buffers.append(sent_dm)
                 self.view.insert_buffer(sent_dm.buffer, name=_(u"Sent direct messages"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
             elif i == 'sent_tweets':
-    sent_tweets = twitterBuffers.baseBufferController(self.view.nb, "get_user_timeline", "sent_tweets", session, session.db["user_name"], screen_name=session.db["user_name"], tweet_mode="extended")
+                sent_tweets = twitterBuffers.baseBufferController(self.view.nb, "user_timeline", "sent_tweets", session, session.db["user_name"], screen_name=session.db["user_name"], tweet_mode="extended")
                 self.buffers.append(sent_tweets)
                 self.view.insert_buffer(sent_tweets.buffer, name=_(u"Sent tweets"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
             elif i == 'favorites':
-    favourites = twitterBuffers.baseBufferController(self.view.nb, "get_favorites", "favourites", session, session.db["user_name"], sound="favourite.ogg", tweet_mode="extended")
+                favourites = twitterBuffers.baseBufferController(self.view.nb, "favorites", "favourites", session, session.db["user_name"], sound="favourite.ogg", tweet_mode="extended")
                 self.buffers.append(favourites)
                 self.view.insert_buffer(favourites.buffer, name=_(u"Likes"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
             elif i == 'followers':
-    followers = twitterBuffers.peopleBufferController(self.view.nb, "get_followers_list", "followers", session, session.db["user_name"], sound="update_followers.ogg", screen_name=session.db["user_name"])
+                followers = twitterBuffers.peopleBufferController(self.view.nb, "followers", "followers", session, session.db["user_name"], sound="update_followers.ogg", screen_name=session.db["user_name"])
                 self.buffers.append(followers)
                 self.view.insert_buffer(followers.buffer, name=_(u"Followers"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
             elif i == 'friends':
-    friends = twitterBuffers.peopleBufferController(self.view.nb, "get_friends_list", "friends", session, session.db["user_name"], screen_name=session.db["user_name"])
+                friends = twitterBuffers.peopleBufferController(self.view.nb, "friends", "friends", session, session.db["user_name"], screen_name=session.db["user_name"])
                 self.buffers.append(friends)
                 self.view.insert_buffer(friends.buffer, name=_(u"Friends"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
             elif i == 'blocks':
-    blocks = twitterBuffers.peopleBufferController(self.view.nb, "list_blocks", "blocked", session, session.db["user_name"])
+                blocks = twitterBuffers.peopleBufferController(self.view.nb, "blocks", "blocked", session, session.db["user_name"])
                 self.buffers.append(blocks)
                 self.view.insert_buffer(blocks.buffer, name=_(u"Blocked users"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
             elif i == 'muted':
-    muted = twitterBuffers.peopleBufferController(self.view.nb, "list_mutes", "muted", session, session.db["user_name"])
+                muted = twitterBuffers.peopleBufferController(self.view.nb, "mutes", "muted", session, session.db["user_name"])
                 self.buffers.append(muted)
                 self.view.insert_buffer(muted.buffer, name=_(u"Muted users"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
         timelines = baseBuffers.emptyPanel(self.view.nb, "timelines", session.db["user_name"])
         self.buffers.append(timelines)
         self.view.insert_buffer(timelines.buffer , name=_(u"Timelines"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
         for i in session.settings["other_buffers"]["timelines"]:
-   tl = twitterBuffers.baseBufferController(self.view.nb, "get_user_timeline", "%s-timeline" % (i,), session, session.db["user_name"], sound="tweet_timeline.ogg", bufferType=None, user_id=i, tweet_mode="extended")
+            tl = twitterBuffers.baseBufferController(self.view.nb, "user_timeline", "%s-timeline" % (i,), session, session.db["user_name"], sound="tweet_timeline.ogg", bufferType=None, user_id=i, tweet_mode="extended")
             self.buffers.append(tl)
             self.view.insert_buffer(tl.buffer, name=_(u"Timeline for {}").format(i,), pos=self.view.search("timelines", session.db["user_name"]))
         favs_timelines = baseBuffers.emptyPanel(self.view.nb, "favs_timelines", session.db["user_name"])
         self.buffers.append(favs_timelines)
         self.view.insert_buffer(favs_timelines.buffer , name=_(u"Likes timelines"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
         for i in session.settings["other_buffers"]["favourites_timelines"]:
-   tl = twitterBuffers.baseBufferController(self.view.nb, "get_favorites", "%s-favorite" % (i,), session, session.db["user_name"], bufferType=None, sound="favourites_timeline_updated.ogg", user_id=i, tweet_mode="extended")
+            tl = twitterBuffers.baseBufferController(self.view.nb, "favorites", "%s-favorite" % (i,), session, session.db["user_name"], bufferType=None, sound="favourites_timeline_updated.ogg", user_id=i, tweet_mode="extended")
             self.buffers.append(tl)
             self.view.insert_buffer(tl.buffer, name=_(u"Likes for {}").format(i,), pos=self.view.search("favs_timelines", session.db["user_name"]))
         followers_timelines = baseBuffers.emptyPanel(self.view.nb, "followers_timelines", session.db["user_name"])
         self.buffers.append(followers_timelines)
         self.view.insert_buffer(followers_timelines.buffer , name=_(u"Followers' Timelines"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
         for i in session.settings["other_buffers"]["followers_timelines"]:
-   tl = twitterBuffers.peopleBufferController(self.view.nb, "get_followers_list", "%s-followers" % (i,), session, session.db["user_name"], sound="new_event.ogg", user_id=i)
+            tl = twitterBuffers.peopleBufferController(self.view.nb, "followers", "%s-followers" % (i,), session, session.db["user_name"], sound="new_event.ogg", user_id=i)
             self.buffers.append(tl)
             self.view.insert_buffer(tl.buffer, name=_(u"Followers for {}").format(i,), pos=self.view.search("followers_timelines", session.db["user_name"]))
         friends_timelines = baseBuffers.emptyPanel(self.view.nb, "friends_timelines", session.db["user_name"])
         self.buffers.append(friends_timelines)
         self.view.insert_buffer(friends_timelines.buffer , name=_(u"Friends' Timelines"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
         for i in session.settings["other_buffers"]["friends_timelines"]:
-   tl = twitterBuffers.peopleBufferController(self.view.nb, "get_friends_list", "%s-friends" % (i,), session, session.db["user_name"], sound="new_event.ogg", user_id=i)
+            tl = twitterBuffers.peopleBufferController(self.view.nb, "friends", "%s-friends" % (i,), session, session.db["user_name"], sound="new_event.ogg", user_id=i)
             self.buffers.append(tl)
             self.view.insert_buffer(tl.buffer, name=_(u"Friends for {}").format(i,), pos=self.view.search("friends_timelines", session.db["user_name"]))
         lists = baseBuffers.emptyPanel(self.view.nb, "lists", session.db["user_name"])
         self.buffers.append(lists)
         self.view.insert_buffer(lists.buffer , name=_(u"Lists"), pos=self.view.search(session.db["user_name"], session.db["user_name"]))
         for i in session.settings["other_buffers"]["lists"]:
-   tl = twitterBuffers.listBufferController(self.view.nb, "get_list_statuses", "%s-list" % (i,), session, session.db["user_name"], bufferType=None, sound="list_tweet.ogg", list_id=utils.find_list(i, session.db["lists"]), tweet_mode="extended")
+            tl = twitterBuffers.listBufferController(self.view.nb, "list_timeline", "%s-list" % (i,), session, session.db["user_name"], bufferType=None, sound="list_tweet.ogg", list_id=utils.find_list(i, session.db["lists"]), tweet_mode="extended")
             session.lists.append(tl)
             self.buffers.append(tl)
             self.view.insert_buffer(tl.buffer, name=_(u"List for {}").format(i), pos=self.view.search("lists", session.db["user_name"]))
@@ -530,11 +526,11 @@ class Controller(object):
         if not hasattr(buff, "get_right_tweet"): return
         tweet = buff.get_right_tweet()
         if buff.type == "people":
-   users = [tweet["screen_name"]]
+            users = [tweet.screen_name]
         elif buff.type == "dm":
-   users = [buff.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]]
+            users = [buff.session.get_user(tweet.message_create["sender_id"]).screen_name]
         else:
-   users = utils.get_all_users(tweet, buff.session.db)
+            users = utils.get_all_users(tweet, buff.session)
         dlg = dialogs.utils.selectUserDialog(_(u"Select the user"), users)
         if dlg.get_response() == widgetUtils.OK:
             user = dlg.get_user()
@@ -547,11 +543,11 @@ class Controller(object):
         if not hasattr(buff, "get_right_tweet"): return
         tweet = buff.get_right_tweet()
         if buff.type == "people":
-   users = [tweet["screen_name"]]
+            users = [tweet.screen_name]
         elif buff.type == "dm":
-   users = [buff.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]]
+            users = [buff.session.get_user(tweet.message_create["sender_id"]).screen_name]
         else:
-   users = utils.get_all_users(tweet, buff.session.db)
+            users = utils.get_all_users(tweet, buff.session)
         dlg = dialogs.utils.selectUserDialog(_(u"Select the user"), users)
         if dlg.get_response() == widgetUtils.OK:
             user = dlg.get_user()
@@ -561,25 +557,25 @@ class Controller(object):
         dlg.populate_list([compose.compose_list(item) for item in buff.session.db["lists"]])
         if dlg.get_response() == widgetUtils.OK:
             try:
-    list = buff.session.twitter.add_list_member(list_id=buff.session.db["lists"][dlg.get_item()]["id"], screen_name=user)
-    older_list = utils.find_item(buff.session.db["lists"][dlg.get_item()]["id"], buff.session.db["lists"])
-    listBuffer = self.search_buffer("%s-list" % (buff.session.db["lists"][dlg.get_item()]["name"].lower()), buff.session.db["user_name"])
+                list = buff.session.twitter.add_list_member(list_id=buff.session.db["lists"][dlg.get_item()].id, screen_name=user)
+                older_list = utils.find_item(buff.session.db["lists"][dlg.get_item()].id, buff.session.db["lists"])
+                listBuffer = self.search_buffer("%s-list" % (buff.session.db["lists"][dlg.get_item()].name.lower()), buff.session.db["user_name"])
                 if listBuffer != None: listBuffer.get_user_ids()
                 buff.session.db["lists"].pop(older_list)
                 buff.session.db["lists"].append(list)
-   except TwythonError as e:
-    output.speak("error %s: %s" % (e.error_code, e.msg))
+            except TweepError as e:
+                output.speak("error %s: %s" % (e.api_code, e.reason))
 
     def remove_from_list(self, *args, **kwargs):
         buff = self.get_best_buffer()
         if not hasattr(buff, "get_right_tweet"): return
         tweet = buff.get_right_tweet()
         if buff.type == "people":
-   users = [tweet["screen_name"]]
+            users = [tweet.screen_name]
         elif buff.type == "dm":
-   users = [buff.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]]
+            users = [buff.session.get_user(tweet.message_create["sender_id"]).screen_name]
         else:
-   users = utils.get_all_users(tweet, buff.session.db)
+            users = utils.get_all_users(tweet, buff.session)
         dlg = dialogs.utils.selectUserDialog(_(u"Select the user"), users)
         if dlg.get_response() == widgetUtils.OK:
             user = dlg.get_user()
@@ -589,14 +585,14 @@ class Controller(object):
         dlg.populate_list([compose.compose_list(item) for item in buff.session.db["lists"]])
         if dlg.get_response() == widgetUtils.OK:
             try:
-    list = buff.session.twitter.delete_list_member(list_id=buff.session.db["lists"][dlg.get_item()]["id"], screen_name=user)
-    older_list = utils.find_item(buff.session.db["lists"][dlg.get_item()]["id"], buff.session.db["lists"])
-    listBuffer = self.search_buffer("%s-list" % (buff.session.db["lists"][dlg.get_item()]["name"].lower()), buff.session.db["user_name"])
+                list = buff.session.twitter.remove_list_member(list_id=buff.session.db["lists"][dlg.get_item()].id, screen_name=user)
+                older_list = utils.find_item(buff.session.db["lists"][dlg.get_item()].id, buff.session.db["lists"])
+                listBuffer = self.search_buffer("%s-list" % (buff.session.db["lists"][dlg.get_item()].name.lower()), buff.session.db["user_name"])
                 if listBuffer != None: listBuffer.get_user_ids()
                 buff.session.db["lists"].pop(older_list)
                 buff.session.db["lists"].append(list)
-   except TwythonError as e:
-    output.speak("error %s: %s" % (e.error_code, e.msg))
+            except TweepError as e:
+                output.speak("error %s: %s" % (e.api_code, e.reason))
 
     def list_manager(self, *args, **kwargs):
         s = self.get_best_buffer().session
@@ -621,6 +617,7 @@ class Controller(object):
             if d.needs_restart == True:
                 commonMessageDialogs.needs_restart()
                 buff.session.settings.write()
+                buff.session.save_persistent_data()
                 restart.restart_program()
 
     def report_error(self, *args, **kwargs):
@@ -655,8 +652,8 @@ class Controller(object):
             if sessions.sessions[item].logged == False: continue
             log.debug("Disconnecting streams for %s session" % (sessions.sessions[item].session_id,))
             sessions.sessions[item].sound.cleaner.cancel()
-   log.debug("Shelving database for " +    sessions.sessions[item].session_id)
-   sessions.sessions[item].shelve()
+            log.debug("Saving database for " +    sessions.sessions[item].session_id)
+            sessions.sessions[item].save_persistent_data()
         if system == "Windows":
             self.systrayIcon.RemoveIcon()
             pidpath = os.path.join(os.getenv("temp"), "{}.pid".format(application.name))
@@ -669,11 +666,11 @@ class Controller(object):
         if not hasattr(buff, "get_right_tweet"): return
         tweet = buff.get_right_tweet()
         if buff.type == "people":
-   users = [tweet["screen_name"]]
+            users = [tweet.screen_name]
         elif buff.type == "dm":
-   users = [buff.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]]
+            users = [buff.session.get_user(tweet.message_create["sender_id"]).screen_name]
         else:
-   users = utils.get_all_users(tweet, buff.session.db)
+            users = utils.get_all_users(tweet, buff.session)
         u = userActionsController.userActionsController(buff, users)
 
     def unfollow(self, *args, **kwargs):
@@ -681,11 +678,11 @@ class Controller(object):
         if not hasattr(buff, "get_right_tweet"): return
         tweet = buff.get_right_tweet()
         if buff.type == "people":
-   users = [tweet["screen_name"]]
+            users = [tweet.screen_name]
         elif buff.type == "dm":
-   users = [buff.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]]
+            users = [buff.session.get_user(tweet.message_create["sender_id"]).screen_name]
         else:
-   users = utils.get_all_users(tweet, buff.session.db)
+            users = utils.get_all_users(tweet, buff.session)
         u = userActionsController.userActionsController(buff, users, "unfollow")
 
     def mute(self, *args, **kwargs):
@@ -693,11 +690,11 @@ class Controller(object):
         if not hasattr(buff, "get_right_tweet"): return
         tweet = buff.get_right_tweet()
         if buff.type == "people":
-   users = [tweet["screen_name"]]
+            users = [tweet.screen_name]
         elif buff.type == "dm":
-   users = [buff.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]]
+            users = [buff.session.get_user(tweet.message_create["sender_id"]).screen_name]
         else:
-   users = utils.get_all_users(tweet, buff.session.db)
+            users = utils.get_all_users(tweet, buff.session)
         u = userActionsController.userActionsController(buff, users, "mute")
 
     def unmute(self, *args, **kwargs):
@@ -705,11 +702,11 @@ class Controller(object):
         if not hasattr(buff, "get_right_tweet"): return
         tweet = buff.get_right_tweet()
         if buff.type == "people":
-   users = [tweet["screen_name"]]
+            users = [tweet.screen_name]
         elif buff.type == "dm":
-   users = [buff.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]]
+            users = [buff.session.get_user(tweet.message_create["sender_id"]).screen_name]
         else:
-   users = utils.get_all_users(tweet, buff.session.db)
+            users = utils.get_all_users(tweet, buff.session)
         u = userActionsController.userActionsController(buff, users, "unmute")
 
     def block(self, *args, **kwargs):
@@ -717,11 +714,11 @@ class Controller(object):
         if not hasattr(buff, "get_right_tweet"): return
         tweet = buff.get_right_tweet()
         if buff.type == "people":
-   users = [tweet["screen_name"]]
+            users = [tweet.screen_name]
         elif buff.type == "dm":
-   users = [buff.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]]
+            users = [buff.session.get_user(tweet.message_create["sender_id"]).screen_name]
         else:
-   users = utils.get_all_users(tweet, buff.session.db)
+            users = utils.get_all_users(tweet, buff.session)
         u = userActionsController.userActionsController(buff, users, "block")
 
     def unblock(self, *args, **kwargs):
@@ -729,11 +726,11 @@ class Controller(object):
         if not hasattr(buff, "get_right_tweet"): return
         tweet = buff.get_right_tweet()
         if buff.type == "people":
-   users = [tweet["screen_name"]]
+            users = [tweet.screen_name]
         elif buff.type == "dm":
-   users = [buff.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]]
+            users = [buff.session.get_user(tweet.message_create["sender_id"]).screen_name]
         else:
-   users = utils.get_all_users(tweet, buff.session.db)
+            users = utils.get_all_users(tweet, buff.session)
         u = userActionsController.userActionsController(buff, users, "unblock")
 
     def report(self, *args, **kwargs):
@@ -741,11 +738,11 @@ class Controller(object):
         if not hasattr(buff, "get_right_tweet"): return
         tweet = buff.get_right_tweet()
         if buff.type == "people":
-   users = [tweet["screen_name"]]
+            users = [tweet.screen_name]
         elif buff.type == "dm":
-   users = [buff.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]]
+            users = [buff.session.get_user(tweet.message_create["sender_id"]).screen_name]
         else:
-   users = utils.get_all_users(tweet, buff.session.db)
+            users = utils.get_all_users(tweet, buff.session)
         u = userActionsController.userActionsController(buff, users, "report")
 
     def post_tweet(self, event=None):
@@ -775,7 +772,7 @@ class Controller(object):
         if buffer.type == "dm" or buffer.type == "people" or buffer.type == "events":
             return
         else:
-   id = buffer.get_tweet()["id"]
+            id = buffer.get_tweet().id
             call_threaded(buffer.session.api_call, call_name="create_favorite", _sound="favourite.ogg", id=id)
 
     def remove_from_favourites(self, *args, **kwargs):
@@ -783,7 +780,7 @@ class Controller(object):
         if buffer.type == "dm" or buffer.type == "people" or buffer.type == "events":
             return
         else:
-   id = buffer.get_tweet()["id"]
+            id = buffer.get_tweet().id
             call_threaded(buffer.session.api_call, call_name="destroy_favorite", id=id)
 
     def toggle_like(self, *args, **kwargs):
@@ -791,9 +788,9 @@ class Controller(object):
         if buffer.type == "dm" or buffer.type == "people" or buffer.type == "events":
             return
         else:
-   id = buffer.get_tweet()["id"]
-   tweet = buffer.session.twitter.show_status(id=id, include_ext_alt_text=True, tweet_mode="extended")
-   if tweet["favorited"] == False:
+            id = buffer.get_tweet().id
+            tweet = buffer.session.twitter.get_status(id=id, include_ext_alt_text=True, tweet_mode="extended")
+            if tweet.favorited == False:
                 call_threaded(buffer.session.api_call, call_name="create_favorite", _sound="favourite.ogg", id=id)
             else:
                 call_threaded(buffer.session.api_call, call_name="destroy_favorite", id=id)
@@ -808,7 +805,7 @@ class Controller(object):
         elif buffer.type == "dm":
             non_tweet = buffer.get_formatted_message()
             item = buffer.get_right_tweet()
-   original_date = arrow.get(int(item["created_timestamp"][:-3]))
+            original_date = arrow.get(int(item.created_timestamp))
             date = original_date.shift(seconds=buffer.session.db["utc_offset"]).format(_(u"MMM D, YYYY. H:m"), locale=languageHandler.getLanguage())
             msg = messages.viewTweet(non_tweet, [], False, date=date)
         else:
@@ -828,11 +825,11 @@ class Controller(object):
         if not hasattr(buff, "get_right_tweet"): return
         tweet = buff.get_right_tweet()
         if buff.type == "people":
-   users = [tweet["screen_name"]]
+            users = [tweet.screen_name]
         elif buff.type == "dm":
-   users = [buff.session.get_user(tweet["message_create"]["sender_id"])["screen_name"]]
+            users = [buff.session.get_user(tweet.message_create["sender_id"]).screen_name]
         else:
-   users = utils.get_all_users(tweet, buff.session.db)
+            users = utils.get_all_users(tweet, buff.session)
         dlg = dialogs.userSelection.selectUserDialog(users=users, default=default)
         if dlg.get_response() == widgetUtils.OK:
             usr = utils.if_user_exists(buff.session.twitter, dlg.get_user())
@@ -840,85 +837,85 @@ class Controller(object):
                 if usr == dlg.get_user():
                     commonMessageDialogs.suspended_user()
                     return
-    if usr["protected"] == True:
-     if usr["following"] == False:
+                if usr.protected == True:
+                    if usr.following == False:
                         commonMessageDialogs.no_following()
                         return
                 tl_type = dlg.get_action()
                 if tl_type  == "tweets":
-     if usr["statuses_count"] == 0:
+                    if usr.statuses_count == 0:
                         commonMessageDialogs.no_tweets()
                         return
-     if usr["id_str"] in buff.session.settings["other_buffers"]["timelines"]:
+                    if usr.id_str in buff.session.settings["other_buffers"]["timelines"]:
                         commonMessageDialogs.timeline_exist()
                         return
-     tl = twitterBuffers.baseBufferController(self.view.nb, "get_user_timeline", "%s-timeline" % (usr["id_str"],), buff.session, buff.session.db["user_name"], bufferType=None, sound="tweet_timeline.ogg", user_id=usr["id_str"], tweet_mode="extended")
+                    tl = twitterBuffers.baseBufferController(self.view.nb, "user_timeline", "%s-timeline" % (usr.id_str,), buff.session, buff.session.db["user_name"], bufferType=None, sound="tweet_timeline.ogg", user_id=usr.id_str, tweet_mode="extended")
                     try:
                         tl.start_stream(play_sound=False)
-     except TwythonAuthError:
+                    except ValueError:
                         commonMessageDialogs.unauthorized()
                         return
                     pos=self.view.search("timelines", buff.session.db["user_name"])
                     self.insert_buffer(tl, pos+1)
                     self.view.insert_buffer(tl.buffer, name=_(u"Timeline for {}").format(dlg.get_user()), pos=pos)
-     buff.session.settings["other_buffers"]["timelines"].append(usr["id_str"])
+                    buff.session.settings["other_buffers"]["timelines"].append(usr.id_str)
                     pub.sendMessage("buffer-title-changed", buffer=tl)
                     buff.session.sound.play("create_timeline.ogg")
                 elif tl_type == "favourites":
-     if usr["favourites_count"] == 0:
+                    if usr.favourites_count == 0:
                         commonMessageDialogs.no_favs()
                         return
-     if usr["id_str"] in buff.session.settings["other_buffers"]["favourites_timelines"]:
+                    if usr.id_str in buff.session.settings["other_buffers"]["favourites_timelines"]:
                         commonMessageDialogs.timeline_exist()
                         return
-     tl = twitterBuffers.baseBufferController(self.view.nb, "get_favorites", "%s-favorite" % (usr["id_str"],), buff.session, buff.session.db["user_name"], bufferType=None, sound="favourites_timeline_updated.ogg", user_id=usr["id_str"], tweet_mode="extended")
+                    tl = twitterBuffers.baseBufferController(self.view.nb, "favorites", "%s-favorite" % (usr.id_str,), buff.session, buff.session.db["user_name"], bufferType=None, sound="favourites_timeline_updated.ogg", user_id=usr.id_str, tweet_mode="extended")
                     try:
                         tl.start_stream(play_sound=False)
-     except TwythonAuthError:
+                    except ValueError:
                         commonMessageDialogs.unauthorized()
                         return
                     pos=self.view.search("favs_timelines", buff.session.db["user_name"])
                     self.insert_buffer(tl, pos+1)
                     self.view.insert_buffer(buffer=tl.buffer, name=_(u"Likes for {}").format(dlg.get_user()), pos=pos)
-     buff.session.settings["other_buffers"]["favourites_timelines"].append(usr["id_str"])
+                    buff.session.settings["other_buffers"]["favourites_timelines"].append(usr.id_str)
                     pub.sendMessage("buffer-title-changed", buffer=buff)
                     buff.session.sound.play("create_timeline.ogg")
                 elif tl_type == "followers":
-     if usr["followers_count"] == 0:
+                    if usr.followers_count == 0:
                         commonMessageDialogs.no_followers()
                         return
-     if usr["id_str"] in buff.session.settings["other_buffers"]["followers_timelines"]:
+                    if usr.id_str in buff.session.settings["other_buffers"]["followers_timelines"]:
                         commonMessageDialogs.timeline_exist()
                         return
-     tl = twitterBuffers.peopleBufferController(self.view.nb, "get_followers_list", "%s-followers" % (usr["id_str"],), buff.session, buff.session.db["user_name"], sound="new_event.ogg", user_id=usr["id_str"])
+                    tl = twitterBuffers.peopleBufferController(self.view.nb, "followers", "%s-followers" % (usr.id_str,), buff.session, buff.session.db["user_name"], sound="new_event.ogg", user_id=usr.id_str)
                     try:
                         tl.start_stream(play_sound=False)
-     except TwythonAuthError:
+                    except ValueError:
                         commonMessageDialogs.unauthorized()
                         return
                     pos=self.view.search("followers_timelines", buff.session.db["user_name"])
                     self.insert_buffer(tl, pos+1)
                     self.view.insert_buffer(buffer=tl.buffer, name=_(u"Followers for {}").format(dlg.get_user()), pos=pos)
-     buff.session.settings["other_buffers"]["followers_timelines"].append(usr["id_str"])
+                    buff.session.settings["other_buffers"]["followers_timelines"].append(usr.id_str)
                     buff.session.sound.play("create_timeline.ogg")
                     pub.sendMessage("buffer-title-changed", buffer=i)
                 elif tl_type == "friends":
-     if usr["friends_count"] == 0:
+                    if usr.friends_count == 0:
                         commonMessageDialogs.no_friends()
                         return
-     if usr["id_str"] in buff.session.settings["other_buffers"]["friends_timelines"]:
+                    if usr.id_str in buff.session.settings["other_buffers"]["friends_timelines"]:
                         commonMessageDialogs.timeline_exist()
                         return
-     tl = twitterBuffers.peopleBufferController(self.view.nb, "get_friends_list", "%s-friends" % (usr["id_str"],), buff.session, buff.session.db["user_name"], sound="new_event.ogg", user_id=usr["id_str"])
+                    tl = twitterBuffers.peopleBufferController(self.view.nb, "friends", "%s-friends" % (usr.id_str,), buff.session, buff.session.db["user_name"], sound="new_event.ogg", user_id=usr.id_str)
                     try:
                         tl.start_stream(play_sound=False)
-     except TwythonAuthError:
+                    except ValueError:
                         commonMessageDialogs.unauthorized()
                         return
                     pos=self.view.search("friends_timelines", buff.session.db["user_name"])
                     self.insert_buffer(tl, pos+1)
                     self.view.insert_buffer(buffer=tl.buffer, name=_(u"Friends for {}").format(dlg.get_user()), pos=pos)
-     buff.session.settings["other_buffers"]["friends_timelines"].append(usr["id_str"])
+                    buff.session.settings["other_buffers"]["friends_timelines"].append(usr.id_str)
                     buff.session.sound.play("create_timeline.ogg")
                     pub.sendMessage("buffer-title-changed", buffer=i)
             else:
@@ -927,8 +924,8 @@ class Controller(object):
 
     def open_conversation(self, *args, **kwargs):
         buffer = self.get_current_buffer()
-  id = buffer.get_right_tweet()["id_str"]
-  user = buffer.get_right_tweet()["user"]["screen_name"]
+        id = buffer.get_right_tweet().id
+        user = buffer.session.get_user(buffer.get_right_tweet().user).screen_name
         search = twitterBuffers.conversationBufferController(self.view.nb, "search", "%s-searchterm" % (id,), buffer.session, buffer.session.db["user_name"], bufferType="searchPanel", sound="search_updated.ogg", since_id=id, q="@{0}".format(user,))
         search.tweet = buffer.get_right_tweet()
         search.start_stream(start=True)
@@ -968,9 +965,9 @@ class Controller(object):
     def reverse_geocode(self, event=None):
         try:
             tweet = self.get_current_buffer().get_tweet()
-   if tweet["coordinates"] != None:
-    x = tweet["coordinates"]["coordinates"][0]
-    y = tweet["coordinates"]["coordinates"][1]
+            if tweet.coordinates != None:
+                x = tweet.coordinates["coordinates"][0]
+                y = tweet.coordinates["coordinates"][1]
                 address = geocoder.reverse_geocode(y, x, language = languageHandler.curLang)
                 if event == None: output.speak(address[0].__str__())
                 else: self.view.show_address(address[0].__str__())
@@ -988,9 +985,9 @@ class Controller(object):
     def view_reverse_geocode(self, event=None):
         try:
             tweet = self.get_current_buffer().get_right_tweet()
-   if tweet["coordinates"] != None:
-    x = tweet["coordinates"]["coordinates"][0]
-    y = tweet["coordinates"]["coordinates"][1]
+            if tweet.coordinates != None:
+                x = tweet.coordinates["coordinates"][0]
+                y = tweet.coordinates["coordinates"][1]
                 address = geocoder.reverse_geocode(y, x, language = languageHandler.curLang)
                 dlg = commonMessageDialogs.view_geodata(address[0].__str__())
             else:
@@ -1278,10 +1275,12 @@ class Controller(object):
         data = buffer.session.check_long_tweet(data)
         if data == False: # Long tweet deleted from twishort.
             return 
+        items = buffer.session.db[buffer.name]
         if buffer.session.settings["general"]["reverse_timelines"] == False:
-   buffer.session.db[buffer.name].append(data)
+            items.append(data)
         else:
-   buffer.session.db[buffer.name].insert(0, data)
+            items.insert(0, data)
+        buffer.session.db[buffer.name] = items
         buffer.add_new_item(data)
 
     def manage_friend(self, data, user):
@@ -1330,7 +1329,11 @@ class Controller(object):
                         i.start_stream()
                     else:
                         i.start_stream(play_sound=False)
-    except TwythonAuthError:
+                except TweepError as err:
+                    log.exception("Error %s starting buffer %s on account %s, with args %r and kwargs %r due to the following reason: %s" % (err.api_code, i.name, i.account, i.args, i.kwargs, err.reason))
+                    # Determine if this error was caused by a block applied to the current user (IE permission errors).
+                    errors_allowed = [130]
+                    if (err.api_code != None and err.api_code not in errors_allowed) or (err.api_code == None and 'Not authorized' in err.reason): # A twitter error, so safely try to remove the buffer.
                         buff = self.view.search(i.name, i.account)
                         i.remove_buffer(force=True)
                         commonMessageDialogs.blocked_timeline()
@@ -1354,34 +1357,34 @@ class Controller(object):
             try:
                 if sessions.sessions[i].is_logged == False: continue
                 sessions.sessions[i].check_connection()
-   except TwythonError: # We shouldn't allow this function to die.
+            except TweepError: # We shouldn't allow this function to die.
                 pass
 
     def create_new_buffer(self, buffer, account, create):
         buff = self.search_buffer("home_timeline", account)
         if create == True:
             if buffer == "favourites":
-    favourites = twitterBuffers.baseBufferController(self.view.nb, "get_favorites", "favourites", buff.session, buff.session.db["user_name"], tweet_mode="extended")
+                favourites = twitterBuffers.baseBufferController(self.view.nb, "favorites", "favourites", buff.session, buff.session.db["user_name"], tweet_mode="extended")
                 self.buffers.append(favourites)
                 self.view.insert_buffer(favourites.buffer, name=_(u"Likes"), pos=self.view.search(buff.session.db["user_name"], buff.session.db["user_name"]))
                 favourites.start_stream(play_sound=False)
             if buffer == "followers":
-    followers = twitterBuffers.peopleBufferController(self.view.nb, "get_followers_list", "followers", buff.session, buff.session.db["user_name"], screen_name=buff.session.db["user_name"])
+                followers = twitterBuffers.peopleBufferController(self.view.nb, "followers", "followers", buff.session, buff.session.db["user_name"], screen_name=buff.session.db["user_name"])
                 self.buffers.append(followers)
                 self.view.insert_buffer(followers.buffer, name=_(u"Followers"), pos=self.view.search(buff.session.db["user_name"], buff.session.db["user_name"]))
                 followers.start_stream(play_sound=False)
             elif buffer == "friends":
-    friends = twitterBuffers.peopleBufferController(self.view.nb, "get_friends_list", "friends", buff.session, buff.session.db["user_name"], screen_name=buff.session.db["user_name"])
+                friends = twitterBuffers.peopleBufferController(self.view.nb, "friends", "friends", buff.session, buff.session.db["user_name"], screen_name=buff.session.db["user_name"])
                 self.buffers.append(friends)
                 self.view.insert_buffer(friends.buffer, name=_(u"Friends"), pos=self.view.search(buff.session.db["user_name"], buff.session.db["user_name"]))
                 friends.start_stream(play_sound=False)
             elif buffer == "blocked":
-    blocks = twitterBuffers.peopleBufferController(self.view.nb, "list_blocks", "blocked", buff.session, buff.session.db["user_name"])
+                blocks = twitterBuffers.peopleBufferController(self.view.nb, "blocks", "blocked", buff.session, buff.session.db["user_name"])
                 self.buffers.append(blocks)
                 self.view.insert_buffer(blocks.buffer, name=_(u"Blocked users"), pos=self.view.search(buff.session.db["user_name"], buff.session.db["user_name"]))
                 blocks.start_stream(play_sound=False)
             elif buffer == "muted":
-    muted = twitterBuffers.peopleBufferController(self.view.nb, "get_muted_users_list", "muted", buff.session, buff.session.db["user_name"])
+                muted = twitterBuffers.peopleBufferController(self.view.nb, "mutes", "muted", buff.session, buff.session.db["user_name"])
                 self.buffers.append(muted)
                 self.view.insert_buffer(muted.buffer, name=_(u"Muted users"), pos=self.view.search(buff.session.db["user_name"], buff.session.db["user_name"]))
                 muted.start_stream(play_sound=False)
@@ -1395,7 +1398,7 @@ class Controller(object):
             if create in buff.session.settings["other_buffers"]["lists"]:
                 output.speak(_(u"This list is already opened"), True)
                 return
-   tl = twitterBuffers.listBufferController(self.view.nb, "get_list_statuses", create+"-list", buff.session, buff.session.db["user_name"], bufferType=None, list_id=utils.find_list(create, buff.session.db["lists"]), tweet_mode="extended")
+            tl = twitterBuffers.listBufferController(self.view.nb, "list_timeline", create+"-list", buff.session, buff.session.db["user_name"], bufferType=None, list_id=utils.find_list(create, buff.session.db["lists"]), tweet_mode="extended")
             buff.session.lists.append(tl)
             pos=self.view.search("lists", buff.session.db["user_name"])
             self.insert_buffer(tl, pos)
@@ -1536,7 +1539,11 @@ class Controller(object):
             if i.session != None and i.session.is_logged == True:
                 try:
                     i.start_stream(mandatory=True)
-    except TwythonAuthError:
+                except TweepError as err:
+                    log.exception("Error %s starting buffer %s on account %s, with args %r and kwargs %r due to the following reason: %s" % (err.api_code, i.name, i.account, i.args, i.kwargs, err.reason))
+                    # Determine if this error was caused by a block applied to the current user (IE permission errors).
+                    errors_allowed = [130]
+                    if (err.api_code != None and err.api_code not in errors_allowed) or (err.api_code == None and 'Not authorized' in err.reason): # A twitter error, so safely try to remove the buffer.
                         buff = self.view.search(i.name, i.account)
                         i.remove_buffer(force=True)
                         commonMessageDialogs.blocked_timeline()
@@ -1576,12 +1583,12 @@ class Controller(object):
             return
         tweet = buffer.get_tweet()
         media_list = []
-  if ("entities" in tweet) and ("media" in tweet["entities"]):
-   [media_list.append(i) for i in tweet["entities"]["media"] if i not in media_list]
-  elif "retweeted_status" in tweet and "media" in tweet["retweeted_status"]["entities"]:
-   [media_list.append(i) for i in tweet["retweeted_status"]["entities"]["media"] if i not in media_list]
-  elif "quoted_status" in tweet and "media" in tweet["quoted_status"]["entities"]:
-   [media_list.append(i) for i in tweet["quoted_status"]["entities"]["media"] if i not in media_list]
+        if hasattr(tweet, "entities") and tweet.entities.get("media") != None:
+            [media_list.append(i) for i in tweet.entities["media"] if i not in media_list]
+        elif hasattr(tweet, "retweeted_status") and tweet.retweeted_status.get("media") != None:
+            [media_list.append(i) for i in tweet.retweeted_status.entities["media"] if i not in media_list]
+        elif hasattr(tweet, "quoted_status") and tweet.quoted_status.entities.get("media") != None:
+            [media_list.append(i) for i in tweet.quoted_status.entities["media"] if i not in media_list]
         if len(media_list) > 1:
             image_list = [_(u"Picture {0}").format(i,) for i in range(0, len(media_list))]
             dialog = dialogs.urlList.urlList(title=_(u"Select the picture"))
@@ -1597,7 +1604,7 @@ class Controller(object):
         if buffer.session.settings["mysc"]["ocr_language"] != "":
             ocr_lang = buffer.session.settings["mysc"]["ocr_language"]
         else:
-   ocr_lang = ocr.OCRSpace.short_langs.index(tweet["lang"])
+            ocr_lang = ocr.OCRSpace.short_langs.index(tweet.lang)
             ocr_lang = ocr.OCRSpace.OcrLangs[ocr_lang]
         api = ocr.OCRSpace.OCRSpaceAPI()
         try:
@@ -1619,4 +1626,4 @@ class Controller(object):
 
     def save_data_in_db(self):
         for i in sessions.sessions:
-   sessions.sessions[i].shelve()
+            sessions.sessions[i].save_persistent_data()

src/controller/messages.py

@@ -1,12 +1,6 @@
 # -*- coding: utf-8 -*-
-from __future__ import absolute_import
-from __future__ import unicode_literals
-from builtins import str
-from builtins import range
-from builtins import object
 import re
 import platform
-from . import attach
 import arrow
 import languageHandler
 system = platform.system()
@@ -16,15 +10,16 @@ import url_shortener
 import sound
 import config
 from pubsub import pub
+from twitter_text import parse_tweet
 if system == "Windows":
     from wxUI.dialogs import message, urlList
     from wxUI import commonMessageDialogs
-
     from extra import translator, SpellChecker, autocompletionUsers
     from extra.AudioUploader import audioUploader
 elif system == "Linux":
     from gtkUI.dialogs import message
 from sessions.twitter import utils
+from . import attach
 
 class basicTweet(object):
     """ This class handles the tweet main features. Other classes should derive from this class."""
@@ -50,8 +45,11 @@ class basicTweet(object):
         dlg = translator.gui.translateDialog()
         if dlg.get_response() == widgetUtils.OK:
             text_to_translate = self.message.get_text()
-   dest = [x[0] for x in translator.translator.available_languages()][dlg.get("dest_lang")]
-   msg = translator.translator.translate(text=text_to_translate, target=dest)
+            language_dict = translator.translator.available_languages()
+            for k in language_dict:
+                if language_dict[k] == dlg.dest_lang.GetStringSelection():
+                    dst = k
+            msg = translator.translator.translate(text=text_to_translate, target=dst)
             self.message.set_text(msg)
             self.text_processor()
             self.message.text_focus()
@@ -105,8 +103,10 @@ class basicTweet(object):
             self.message.disable_button("shortenButton")
             self.message.disable_button("unshortenButton")
         if self.message.get("long_tweet") == False:
-   self.message.set_title(_(u"%s - %s of %d characters") % (self.title, len(self.message.get_text()), self.max))
-   if len(self.message.get_text()) > self.max:
+            text = self.message.get_text()
+            results = parse_tweet(text)
+            self.message.set_title(_(u"%s - %s of %d characters") % (self.title, results.weightedLength, self.max))
+            if results.weightedLength > self.max:
                 self.session.sound.play("max_length.ogg")
         else:
             self.message.set_title(_(u"%s - %s characters") % (self.title, len(self.message.get_text())))
@@ -194,6 +194,11 @@ class dm(basicTweet):
         super(dm, self).__init__(session, title, caption, text, messageType="dm", max=10000)
         widgetUtils.connect_event(self.message.autocompletionButton, widgetUtils.BUTTON_PRESSED, self.autocomplete_users)
         self.text_processor()
+        widgetUtils.connect_event(self.message.cb, widgetUtils.ENTERED_TEXT, self.user_changed)
+
+    def user_changed(self, *args, **kwargs):
+        self.title = _("Direct message to %s") % (self.message.get_user())
+        self.text_processor()
 
     def autocomplete_users(self, *args, **kwargs):
         c = autocompletionUsers.completion.autocompletionUsers(self.message, self.session.session_id)
@@ -211,56 +216,57 @@ class viewTweet(basicTweet):
             text = ""
             for i in range(0, len(tweetList)):
                 # tweets with message keys are longer tweets, the message value is the full messaje taken from twishort.
-    if "message" in tweetList[i] and tweetList[i]["is_quote_status"] == False:
+                if hasattr(tweetList[i], "message")  and tweetList[i].is_quote_status == False:
                     value = "message"
                 else:
                     value = "full_text"
-    if "retweeted_status" in tweetList[i] and tweetList[i]["is_quote_status"] == False:
-     if ("message" in tweetList[i]) == False:
-      text = text + "rt @%s: %s\n" % (tweetList[i]["retweeted_status"]["user"]["screen_name"], tweetList[i]["retweeted_status"]["full_text"])
+                if hasattr(tweetList[i], "retweeted_status") and tweetList[i].is_quote_status == False:
+                    if not hasattr(tweetList[i], "message"):
+                        text = text + "rt @%s: %s\n" % (tweetList[i].retweeted_status.user.screen_name, tweetList[i].retweeted_status.full_text)
                     else:
-      text = text + "rt @%s: %s\n" % (tweetList[i]["retweeted_status"]["user"]["screen_name"], tweetList[i][value])
+                        text = text + "rt @%s: %s\n" % (tweetList[i].retweeted_status.user.screen_name, getattr(tweetList[i], value))
                 else:
-     text = text + " @%s: %s\n" % (tweetList[i]["user"]["screen_name"], tweetList[i][value])
+                    text = text + " @%s: %s\n" % (tweetList[i].user.screen_name, getattr(tweetList[i], value))
                 # tweets with extended_entities could include image descriptions.
-    if "extended_entities" in tweetList[i] and "media" in tweetList[i]["extended_entities"]:
-     for z in tweetList[i]["extended_entities"]["media"]:
+                if hasattr(tweetList[i], "extended_entities") and "media" in tweetList[i].extended_entities:
+                    for z in tweetList[i].extended_entities["media"]:
                         if "ext_alt_text" in z and z["ext_alt_text"] != None:
                             image_description.append(z["ext_alt_text"])
-    if "retweeted_status" in tweetList[i] and "extended_entities" in tweetList[i]["retweeted_status"] and "media" in tweetList[i]["retweeted_status"]["extended_entities"]:
-     for z in tweetList[i]["retweeted_status"]["extended_entities"]["media"]:
+                if hasattr(tweetList[i], "retweeted_status") and hasattr(tweetList[i].retweeted_status, "extended_entities") and "media" in tweetList[i].retweeted_status["extended_entities"]:
+                    for z in tweetList[i].retweeted_status.extended_entities["media"]:
                         if "ext_alt_text" in z and z["ext_alt_text"] != None:
                             image_description.append(z["ext_alt_text"])
             # set rt and likes counters.
-   rt_count = str(tweet["retweet_count"])
-   favs_count = str(tweet["favorite_count"])
+            rt_count = str(tweet.retweet_count)
+            favs_count = str(tweet.favorite_count)
             # Gets the client from where this tweet was made.
-   source = re.sub(r"(?s)<.*?>", "", tweet["source"])
-   original_date = arrow.get(tweet["created_at"], "ddd MMM DD H:m:s Z YYYY", locale="en")
+            source = tweet.source
+            original_date = arrow.get(tweet.created_at, locale="en")
             date = original_date.shift(seconds=utc_offset).format(_(u"MMM D, YYYY. H:m"), locale=languageHandler.getLanguage())
             if text == "":
-    if "message" in tweet:
+                if hasattr(tweet, "message"):
                     value = "message"
                 else:
                     value = "full_text"
-    if "retweeted_status" in tweet:
-     if ("message" in tweet) == False:
-      text = "rt @%s: %s" % (tweet["retweeted_status"]["user"]["screen_name"], tweet["retweeted_status"]["full_text"])
+                if hasattr(tweet, "retweeted_status"):
+                    if not hasattr(tweet, "message"):
+                        text = "rt @%s: %s" % (tweet.retweeted_status.user.screen_name, tweet.retweeted_status.full_text)
                     else:
-      text = "rt @%s: %s" % (tweet["retweeted_status"]["user"]["screen_name"], tweet[value])
+                        text = "rt @%s: %s" % (tweet.retweeted_status.user.screen_name, getattr(tweet, value))
                 else:
-     text = tweet[value]
+                    text = getattr(tweet, value)
             text = self.clear_text(text)
-   if "extended_entities" in tweet and "media" in tweet["extended_entities"]:
-    for z in tweet["extended_entities"]["media"]:
+            if hasattr(tweet, "extended_entities") and "media" in tweet.extended_entities:
+                for z in tweet.extended_entities["media"]:
                     if "ext_alt_text" in z and z["ext_alt_text"] != None:
                         image_description.append(z["ext_alt_text"])
-   if "retweeted_status" in tweet and "extended_entities" in tweet["retweeted_status"] and "media" in tweet["retweeted_status"]["extended_entities"]:
-    for z in tweet["retweeted_status"]["extended_entities"]["media"]:
+            if hasattr(tweet, "retweeted_status") and hasattr(tweet.retweeted_status, "extended_entities") and "media" in tweet.retweeted_status.extended_entities:
+                for z in tweet.retweeted_status.extended_entities["media"]:
                     if "ext_alt_text" in z and z["ext_alt_text"] != None:
                         image_description.append(z["ext_alt_text"])
             self.message = message.viewTweet(text, rt_count, favs_count, source, date)
-   self.message.set_title(len(text))
+            results = parse_tweet(text)
+            self.message.set_title(results.weightedLength)
             [self.message.set_image_description(i) for i in image_description]
         else:
             self.title = _(u"View item")

src/controller/settings.py

@@ -1,7 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
-from builtins import str
-from builtins import object
 import os
 import webbrowser
 import sound_lib
@@ -74,12 +71,12 @@ class globalSettingsController(object):
         self.dialog.set_value("general", "update_period", config.app["app-settings"]["update_period"])
         self.dialog.set_value("general", "check_for_updates", config.app["app-settings"]["check_for_updates"])
         self.dialog.set_value("general", "remember_mention_and_longtweet", config.app["app-settings"]["remember_mention_and_longtweet"])
-  proxyTypes=config.proxyTypes
-  self.dialog.create_proxy([_(u"Direct connection")]+proxyTypes)
-  if config.app["proxy"]["type"] not in proxyTypes:
+        proxyTypes = [_("System default"), _("HTTP"), _("SOCKS v4"), _("SOCKS v4 with DNS support"), _("SOCKS v5"), _("SOCKS v5 with DNS support")]
+        self.dialog.create_proxy(proxyTypes)
+        try:
+            self.dialog.proxy.type.SetSelection(config.app["proxy"]["type"])
+        except:
             self.dialog.proxy.type.SetSelection(0)
-  else:
-   self.dialog.proxy.type.SetSelection(proxyTypes.index(config.app["proxy"]["type"])+1)
         self.dialog.set_value("proxy", "server", config.app["proxy"]["server"])
         self.dialog.set_value("proxy", "port", config.app["proxy"]["port"])
         self.dialog.set_value("proxy", "user", config.app["proxy"]["user"])
@@ -121,7 +118,7 @@ class globalSettingsController(object):
         if config.app["proxy"]["type"]!=self.dialog.get_value("proxy", "type") or config.app["proxy"]["server"] != self.dialog.get_value("proxy", "server") or config.app["proxy"]["port"] != self.dialog.get_value("proxy", "port") or config.app["proxy"]["user"] != self.dialog.get_value("proxy", "user") or config.app["proxy"]["password"] != self.dialog.get_value("proxy", "password"):
             if self.is_started == True:
                 self.needs_restart = True
-   config.app["proxy"]["type"]=self.dialog.get_value("proxy", "type")
+            config.app["proxy"]["type"] = self.dialog.proxy.type.Selection
             config.app["proxy"]["server"] = self.dialog.get_value("proxy", "server")
             config.app["proxy"]["port"] = self.dialog.get_value("proxy", "port")
             config.app["proxy"]["user"] = self.dialog.get_value("proxy", "user")
@@ -151,6 +148,7 @@ class accountSettingsController(globalSettingsController):
         else:
             self.dialog.set_value("general", "retweet_mode", _(u"Retweet with comments"))
         self.dialog.set_value("general", "persist_size", str(self.config["general"]["persist_size"]))
+        self.dialog.set_value("general", "load_cache_in_memory", self.config["general"]["load_cache_in_memory"])
         self.dialog.create_reporting()
         self.dialog.set_value("reporting", "speech_reporting", self.config["reporting"]["speech_reporting"])
         self.dialog.set_value("reporting", "braille_reporting", self.config["reporting"]["braille_reporting"])
@@ -193,6 +191,9 @@ class accountSettingsController(globalSettingsController):
             self.config["general"]["relative_times"] = self.dialog.get_value("general", "relative_time")
         self.config["general"]["show_screen_names"] = self.dialog.get_value("general", "show_screen_names")
         self.config["general"]["max_tweets_per_call"] = self.dialog.get_value("general", "itemsPerApiCall")
+        if self.config["general"]["load_cache_in_memory"] != self.dialog.get_value("general", "load_cache_in_memory"):
+            self.config["general"]["load_cache_in_memory"] = self.dialog.get_value("general", "load_cache_in_memory")
+            self.needs_restart = True
         if self.config["general"]["persist_size"] != self.dialog.get_value("general", "persist_size"):
             if self.dialog.get_value("general", "persist_size") == '':
                 self.config["general"]["persist_size"] =-1

src/controller/trendingTopics.py

@@ -1,6 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
-from builtins import object
 from wxUI.dialogs import trends
 import widgetUtils
 
@@ -10,7 +8,7 @@ class trendingTopicsController(object):
         self.countries = {}
         self.cities = {}
         self.dialog = trends.trendingTopicsDialog()
-  self.information = session.twitter.get_available_trends()
+        self.information = session.twitter.trends_available()
         self.split_information()
         widgetUtils.connect_event(self.dialog.country, widgetUtils.RADIOBUTTON, self.get_places)
         widgetUtils.connect_event(self.dialog.city, widgetUtils.RADIOBUTTON, self.get_places)

src/controller/user.py

@@ -1,6 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
-from builtins import object
 import wx
 import webbrowser
 import widgetUtils
@@ -8,7 +6,8 @@ import output
 from wxUI.dialogs import update_profile, show_user
 import logging
 log = logging.getLogger("controller.user")
-from twython import TwythonError
+from tweepy.error import TweepError
+from sessions.twitter import utils
 
 class profileController(object):
     def __init__(self, session, user=None):
@@ -25,36 +24,36 @@ class profileController(object):
         else:
             try:
                 self.get_data(screen_name=self.user)
-   except TwythonError as err:
-    if err.error_code == 404:
+            except TweepError as err:
+                if err.api_code == 50:
                     wx.MessageDialog(None, _(u"That user does not exist"), _(u"Error"), wx.ICON_ERROR).ShowModal()
-    if err.error_code == 403:
+                if err.api_code == 403:
                     wx.MessageDialog(None, _(u"User has been suspended"), _(u"Error"), wx.ICON_ERROR).ShowModal()
-    log.error("error %d: %s" % (err.error_code, err.msg))
+                log.error("error %d: %s" % (err.api_code, err.reason))
                 return
             self.dialog = show_user.showUserProfile()
             string = self.get_user_info()
             self.dialog.set("text", string)
-   self.dialog.set_title(_(u"Information for %s") % (self.data["screen_name"]))
-   if self.data["url"] != None:
+            self.dialog.set_title(_(u"Information for %s") % (self.data.screen_name))
+            if self.data.url != None:
                 self.dialog.enable_url()
                 widgetUtils.connect_event(self.dialog.url, widgetUtils.BUTTON_PRESSED, self.visit_url)
         if self.dialog.get_response() == widgetUtils.OK and self.user == None:
             self.do_update()
 
     def get_data(self, screen_name):
-  self.data = self.session.twitter.show_user(screen_name=screen_name)
+        self.data = self.session.twitter.get_user(screen_name=screen_name)
         if screen_name != self.session.db["user_name"]:
             self.friendship_status = self.session.twitter.show_friendship(source_screen_name=self.session.db["user_name"], target_screen_name=screen_name)
 
     def fill_profile_fields(self):
-  self.dialog.set_name(self.data["name"])
-  if self.data["url"] != None:
-   self.dialog.set_url(self.data["url"])
-  if len(self.data["location"]) > 0:
-   self.dialog.set_location(self.data["location"])
-  if len(self.data["description"]) > 0:
-   self.dialog.set_description(self.data["description"])
+        self.dialog.set_name(self.data.name)
+        if self.data.url != None:
+            self.dialog.set_url(self.data.url)
+        if len(self.data.location) > 0:
+            self.dialog.set_location(self.data.location)
+        if len(self.data.description) > 0:
+            self.dialog.set_description(self.data.description)
 
     def get_image(self):
         file = self.dialog.upload_picture()
@@ -84,45 +83,46 @@ class profileController(object):
         if self.file != None:
             try:
                 self.session.twitter.update_profile_image(image=self.file)
-   except TwythonError as e:
-    output.speak(u"Error %s. %s" % (e.error_code, e.msg))
+            except TweepError as e:
+                output.speak(u"Error %s. %s" % (e.api_code, e.reason))
         try:
             self.session.twitter.update_profile(name=name, description=description, location=location, url=url)
-  except TwythonError as e:
-   output.speak(u"Error %s. %s" % (e.error_code, e.msg))
+        except TweepError as e:
+            output.speak(u"Error %s. %s" % (e.api_code, e.reason))
 
     def get_user_info(self):
- 
         string = u""
-  string = string + _(u"Username: @%s\n") % (self.data["screen_name"])
-  string = string + _(u"Name: %s\n") % (self.data["name"])
-  if self.data["location"] != "":
-   string = string + _(u"Location: %s\n") % (self.data["location"])
-  if self.data["url"] != None:
-   string = string+ _(u"URL: %s\n") % (self.data["url"])
-  if self.data["description"] != "":
-   string = string+ _(u"Bio: %s\n") % (self.data["description"])
-  if self.data["protected"] == True: protected = _(u"Yes")
+        string = string + _(u"Username: @%s\n") % (self.data.screen_name)
+        string = string + _(u"Name: %s\n") % (self.data.name)
+        if self.data.location != "":
+            string = string + _(u"Location: %s\n") % (self.data.location)
+        if self.data.url != None:
+            string = string+ _(u"URL: %s\n") % (self.data.entities["url"]["urls"][0]["expanded_url"])
+        if self.data.description != "":
+            if self.data.entities.get("description") != None and self.data.entities["description"].get("urls"):
+                self.data.description = utils.expand_urls(self.data.description, self.data.entities["description"])
+            string = string+ _(u"Bio: %s\n") % (self.data.description)
+        if self.data.protected == True: protected = _(u"Yes")
         else: protected = _(u"No")
         string = string+ _(u"Protected: %s\n") % (protected)
         if hasattr(self, "friendship_status"):
             relation = False
             friendship = "Relationship: "
-   if self.friendship_status["relationship"]["target"]["followed_by"]:
-    friendship += _(u"You follow {0}. ").format(self.data["name"],)
+            if self.friendship_status[0].following:
+                friendship += _(u"You follow {0}. ").format(self.data.name,)
                 relation = True
-   if self.friendship_status["relationship"]["target"]["following"]:
-    friendship += _(u"{0} is following you.").format(self.data["name"],)
+            if self.friendship_status[1].following:
+                friendship += _(u"{0} is following you.").format(self.data.name,)
                 relation = True
             if relation == True:
                 string = string+friendship+"\n"
-  string = string+_(u"Followers: %s\n Friends: %s\n") % (self.data["followers_count"], self.data["friends_count"])
-  if self.data["verified"] == True: verified = _(u"Yes")
+        string = string+_(u"Followers: %s\n Friends: %s\n") % (self.data.followers_count, self.data.friends_count)
+        if self.data.verified == True: verified = _(u"Yes")
         else: verified = _(u"No")
         string = string+ _(u"Verified: %s\n") % (verified)
-  string = string+ _(u"Tweets: %s\n") % (self.data["statuses_count"])
-  string = string+ _(u"Likes: %s") % (self.data["favourites_count"])
+        string = string+ _(u"Tweets: %s\n") % (self.data.statuses_count)
+        string = string+ _(u"Likes: %s") % (self.data.favourites_count)
         return string
 
     def visit_url(self, *args, **kwargs):
-  webbrowser.open_new_tab(self.data["url"])
+        webbrowser.open_new_tab(self.data.url)

src/controller/userActionsController.py

@@ -1,12 +1,9 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
-from builtins import object
-import re
 import widgetUtils
 import output
 from wxUI.dialogs import userActions
 from pubsub import pub
-from twython import TwythonError
+from tweepy.error  import TweepError
 from extra import autocompletionUsers
 
 class userActionsController(object):
@@ -32,51 +29,51 @@ class userActionsController(object):
     def follow(self, user):
         try:
             self.session.twitter.create_friendship(screen_name=user )
-  except TwythonError as err:
-   output.speak("Error %s: %s" % (err.error_code, err.msg), True)
+        except TweepError as err:
+            output.speak("Error %s: %s" % (err.api_code, err.reason), True)
 
     def unfollow(self, user):
         try:
             id = self.session.twitter.destroy_friendship(screen_name=user )
-  except TwythonError as err:
-   output.speak("Error %s: %s" % (err.error_code, err.msg), True)
+        except TweepError as err:
+            output.speak("Error %s: %s" % (err.api_code, err.reason), True)
 
     def mute(self, user):
         try:
             id = self.session.twitter.create_mute(screen_name=user )
-  except TwythonError as err:
-   output.speak("Error %s: %s" % (err.error_code, err.msg), True)
+        except TweepError as err:
+            output.speak("Error %s: %s" % (err.api_code, err.reason), True)
 
     def unmute(self, user):
         try:
             id = self.session.twitter.destroy_mute(screen_name=user )
-  except TwythonError as err:
-   output.speak("Error %s: %s" % (err.error_code, err.msg), True)
+        except TweepError as err:
+            output.speak("Error %s: %s" % (err.api_code, err.reason), True)
 
     def report(self, user):
         try:
             id = self.session.twitter.report_spam(screen_name=user )
-  except TwythonError as err:
-   output.speak("Error %s: %s" % (err.error_code, err.msg), True)
+        except TweepError as err:
+            output.speak("Error %s: %s" % (err.api_code, err.reason), True)
 
     def block(self, user):
         try:
             id = self.session.twitter.create_block(screen_name=user )
-  except TwythonError as err:
-   output.speak("Error %s: %s" % (err.error_code, err.msg), True)
+        except TweepError as err:
+            output.speak("Error %s: %s" % (err.api_code, err.reason), True)
 
     def unblock(self, user):
         try:
             id = self.session.twitter.destroy_block(screen_name=user )
-  except TwythonError as err:
-   output.speak("Error %s: %s" % (err.error_code, err.msg), True)
+        except TweepError as err:
+            output.speak("Error %s: %s" % (err.api_code, err.reason), True)
 
     def ignore_client(self, user):
         tweet = self.buffer.get_right_tweet()
-  if "sender" in tweet:
+        if hasattr(tweet, "sender"):
             output.speak(_(u"You can't ignore direct messages"))
             return
-  client = re.sub(r"(?s)<.*?>", "", tweet["source"])
+        client = tweet.source
         if client not in self.session.settings["twitter"]["ignored_clients"]:
             self.session.settings["twitter"]["ignored_clients"].append(client)
             self.session.settings.write()

src/extra/autocompletionUsers/completion.py

@@ -1,7 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import absolute_import
-from __future__ import unicode_literals
-from builtins import object
 import output
 from . import storage
 from . import wx_menu

src/extra/autocompletionUsers/manage.py

@@ -1,11 +1,6 @@
 # -*- coding: utf-8 -*-
-from __future__ import absolute_import
-from __future__ import unicode_literals
-# -*- coding: utf-8 -*-
-from builtins import object
-from . import storage
 import widgetUtils
-from . import wx_manage
+from . import storage, wx_manage
 from wxUI import commonMessageDialogs
 
 class autocompletionManage(object):
@@ -32,11 +27,11 @@ class autocompletionManage(object):
         if usr == False:
             return
         try:
-   data = self.session.twitter.twitter.show_user(screen_name=usr)
+            data = self.session.twitter.twitter.get_user(screen_name=usr)
         except:
             self.dialog.show_invalid_user_error()
             return
-  self.database.set_user(data["screen_name"], data["name"], 0)
+        self.database.set_user(data.screen_name, data.name, 0)
         self.update_list()
 
     def remove_user(self, ev):

src/extra/autocompletionUsers/settings.py

@@ -1,13 +1,9 @@
 # -*- coding: utf-8 -*-
-from __future__ import absolute_import
-from __future__ import unicode_literals
-# -*- coding: utf-8 -*-
-from builtins import object
-from . import storage
 import widgetUtils
+import output
 from . import wx_settings
 from . import manage
-import output
+from . import storage
 from mysc.thread_utils import call_threaded
 
 class autocompletionSettings(object):
@@ -30,14 +26,14 @@ class autocompletionSettings(object):
         database = storage.storage(self.buffer.session.session_id)
         if self.dialog.get("followers_buffer") == True:
             buffer = self.window.search_buffer("followers", self.config["twitter"]["user_name"])
-   for i in buffer.session.db[buffer.name]["items"]:
-    database.set_user(i["screen_name"], i["name"], 1)
+            for i in buffer.session.db[buffer.name]:
+                database.set_user(i.screen_name, i.name, 1)
         else:
             database.remove_by_buffer(1)
         if self.dialog.get("friends_buffer") == True:
             buffer = self.window.search_buffer("friends", self.config["twitter"]["user_name"])
-   for i in buffer.session.db[buffer.name]["items"]:
-    database.set_user(i["screen_name"], i["name"], 2)
+            for i in buffer.session.db[buffer.name]:
+                database.set_user(i.screen_name, i.name, 2)
         else:
             database.remove_by_buffer(2)
         wx_settings.show_success_dialog()
@@ -52,12 +48,12 @@ def execute_at_startup(window, buffer, config):
     if config["mysc"]["save_followers_in_autocompletion_db"] == True and config["other_buffers"]["show_followers"] == True:
         buffer = window.search_buffer("followers", config["twitter"]["user_name"])
         for i in buffer.session.db[buffer.name]:
-   database.set_user(i["screen_name"], i["name"], 1)
+            database.set_user(i.screen_name, i.name, 1)
     else:
         database.remove_by_buffer(1)
     if config["mysc"]["save_friends_in_autocompletion_db"] == True and config["other_buffers"]["show_friends"] == True:
         buffer = window.search_buffer("friends", config["twitter"]["user_name"])
         for i in buffer.session.db[buffer.name]:
-   database.set_user(i["screen_name"], i["name"], 2)
+            database.set_user(i.screen_name, i.name, 2)
     else:
         database.remove_by_buffer(2)  

src/extra/autocompletionUsers/storage.py

@@ -1,11 +1,9 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
-from builtins import object
-import sqlite3, paths
+import os, sqlite3, paths
 
 class storage(object):
     def __init__(self, session_id):
-  self.connection = sqlite3.connect(paths.config_path("%s/autocompletionUsers.dat" % (session_id)))
+        self.connection = sqlite3.connect(os.path.join(paths.config_path(), "%s/autocompletionUsers.dat" % (session_id)))
         self.cursor = self.connection.cursor()
         if self.table_exist("users") == False:
             self.create_table()

src/extra/autocompletionUsers/wx_manage.py

@@ -1,9 +1,9 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
 import wx
 import widgetUtils
 from multiplatform_widgets import widgets
 import application
+
 class autocompletionManageDialog(widgetUtils.BaseDialog):
     def __init__(self):
         super(autocompletionManageDialog, self).__init__(parent=None, id=-1, title=_(u"Manage Autocompletion database"))

src/extra/autocompletionUsers/wx_menu.py

@@ -1,5 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
 import wx
 
 class menu(wx.Menu):

src/extra/autocompletionUsers/wx_settings.py

@@ -1,5 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
 import wx
 import widgetUtils
 import application

src/extra/translator/translator.py

@@ -1,15 +1,23 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
-from builtins import zip
-from yandex_translate import YandexTranslate
+import logging
+from googletrans import Translator, LANGUAGES
+
+log = logging.getLogger("extras.translator")
+
+# create a single translator instance
+# see https://github.com/ssut/py-googletrans/issues/234
+t = None
 
 def translate(text="", target="en"):
-	t = YandexTranslate("trnsl.1.1.20161012T134532Z.d01b9c75fc39aa74.7d1be75a5166a80583eeb020e10f584168da6bf7")
-	vars = dict(text=text, lang=target)
-	return t.translate(**vars)["text"][0]
+    global t
+    log.debug("Received translation request for language %s, text=%s" % (target, text))
+    if t == None:
+        t = Translator()
+    vars = dict(text=text, dest=target)
+    return t.translate(**vars).text
 
 supported_langs = None
-d = None
+
 languages = {
     "af": _(u"Afrikaans"),
     "sq": _(u"Albanian"),
@@ -105,11 +113,4 @@ languages = {
 }
 
 def available_languages():
-	global supported_langs, d
-	if supported_langs == None and d == None:
-		t = YandexTranslate("trnsl.1.1.20161012T134532Z.d01b9c75fc39aa74.7d1be75a5166a80583eeb020e10f584168da6bf7")
-		supported_langs = t.langs
-		d = []
-		for i in supported_langs:
-			d.append(languages[i])
-	return sorted(zip(supported_langs, d))
+    return dict(sorted(languages.items(), key=lambda x: x[1]))

src/extra/translator/wx_ui.py

@@ -16,37 +16,21 @@
 #    along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 ############################################################
-from __future__ import absolute_import
-from __future__ import unicode_literals
-# -*- coding: utf-8 -*-
-############################################################
-#    Copyright (c) 2013, 2014 Manuel Eduardo Cortéz Vallejo <manuel@manuelcortez.net>
-#       
-#    This program is free software: you can redistribute it and/or modify
-#    it under the terms of the GNU General Public License as published by
-#    the Free Software Foundation, either version 2 of the License, or
-#    (at your option) any later version.
-#
-#    This program is distributed in the hope that it will be useful,
-#    but WITHOUT ANY WARRANTY; without even the implied warranty of
-#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-#    GNU General Public License for more details.
-#
-#    You should have received a copy of the GNU General Public License
-#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
-#
-############################################################
 from . import translator
 import wx
 from wxUI.dialogs import baseDialog
 
 class translateDialog(baseDialog.BaseWXDialog):
     def __init__(self):
+        languages = []
+        language_dict = translator.available_languages()
+        for k in language_dict:
+            languages.append(language_dict[k])
         super(translateDialog, self).__init__(None, -1, title=_(u"Translate message"))
         panel = wx.Panel(self)
         sizer = wx.BoxSizer(wx.VERTICAL)
         staticDest = wx.StaticText(panel, -1, _(u"Target language"))
-  self.dest_lang = wx.ComboBox(panel, -1, choices=[x[1] for x in translator.available_languages()], style = wx.CB_READONLY)
+        self.dest_lang = wx.ComboBox(panel, -1, choices=languages, style = wx.CB_READONLY)
         self.dest_lang.SetFocus()
         self.dest_lang.SetSelection(0)
         listSizer = wx.BoxSizer(wx.HORIZONTAL)

src/fixes/fix_arrow.py

@@ -1,5 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
 from arrow import locales
 from arrow.locales import Locale
 
@@ -8,88 +7,43 @@ def fix():
     locales.get_locale = get_locale
 
 def get_locale(name):
-	locale_cls = locales._locales.get(name.lower())
+    locale_cls = locales._locale_map.get(name.lower())
     if locale_cls is None:
         name = name[:2]
-		locale_cls = locales._locales.get(name.lower())
+        locale_cls = locales._locale_map.get(name.lower())
         if locale_cls == None:
             return locales.EnglishLocale()
     return locale_cls()
 
-class CatalaLocale(Locale):
-	names = ['ca', 'ca_es', 'ca_ca']
-	past = 'Fa {0}'
-	future = '{0}' # I don't know what's the right phrase in catala for the future.
-
-	timeframes = {
-		'now': 'Ara mateix',
-		'seconds': 'segons',
-		'minute': '1 minut',
-		'minutes': '{0} minuts',
-		'hour': 'una hora',
-		'hours': '{0} hores',
-		'day': 'un dia',
-		'days': '{0} dies',
-		'month': 'un mes',
-		'months': '{0} messos',
-		'year': 'un any',
-		'years': '{0} anys',
-	}
-
-	month_names = ['', 'Jener', 'Febrer', 'Març', 'Abril', 'Maig', 'Juny', 'Juliol', 'Agost', 'Setembre', 'Octubre', 'Novembre', 'Decembre']
-	month_abbreviations = ['', 'Jener', 'Febrer', 'Març', 'Abril', 'Maig', 'Juny', 'Juliol', 'Agost', 'Setembre', 'Octubre', 'Novembre', 'Decembre']
-	day_names = ['', 'Dilluns', 'Dimars', 'Dimecres', 'Dijous', 'Divendres', 'Disabte', 'Diumenge']
-	day_abbreviations = ['', 'Dilluns', 'Dimars', 'Dimecres', 'Dijous', 'Divendres', 'Disabte', 'Diumenge']
-
-class GalicianLocale(Locale):
+class GalicianLocale(object):
     names = ['gl', 'gl_es', 'gl_gl']
-	past = 'Fai {0}'
+    past = 'Hai {0}'
     future = 'En {0}'
+    and_word = "e"
 
     timeframes = {
-		'now': 'Agora mesmo',
-		'seconds': 'segundos',
+        'now': 'Agora',
+        "second": "un segundo",
+        'seconds': '{0} segundos',
         'minute': 'un minuto',
         'minutes': '{0} minutos',
-		'hour': 'una hora',
+        'hour': 'unha hora',
         'hours': '{0} horas',
         'day': 'un día',
         'days': '{0} días',
+        "week": "unha semana",
+        "weeks": "{0} semanas",
         'month': 'un mes',
         'months': '{0} meses',
         'year': 'un ano',
         'years': '{0} anos',
     }
 
-	month_names = ['', 'Xaneiro', 'Febreiro', 'Marzo', 'Abril', 'Maio', 'Xuño', 'Xullo', 'Agosto', 'Setembro', 'Outubro', 'Novembro', 'Decembro']
-	month_abbreviations = ['', 'Xan', 'Feb', 'Mar', 'Abr', 'Mai', 'Xun', 'Xul', 'Ago', 'Set', 'Out', 'Nov', 'Dec']
-	day_names = ['', 'Luns', 'Martes', 'Mércores', 'Xoves', 'Venres', 'Sábado', 'Domingo']
-	day_abbreviations = ['', 'Lun', 'Mar', 'Mer', 'xov', 'Ven' 'Sab', 'Dom']
+    meridians = {"am": "am", "pm": "pm", "AM": "AM", "PM": "PM"}
 
-class BasqueLocale(Locale):
-	names = ['eu', 'eu_es', 'eu_eu']
-	past = 'duela {0}'
-	future = '{0} igarota'
-
-	timeframes = {
-		'now': 'Orain',
-		# 'second': 'segundu bat',
-		'seconds': 'segundu batzuk', # without specifying a number.
-		#'seconds':  '{0} segundu', # specifying a number
-		'minute': 'minutu bat',
-		'minutes': '{0} minutu',
-		'hour': 'ordu bat',
-		'hours': '{0} ordu',
-		'day': 'egun bat',
-		'days': '{0} egun',
-		'month': 'hilabete bat',
-		'months': '{0} hilabete',
-		'year': 'urte bat',
-		'years': '{0} urte',
-	}
-
-	month_names = ['', 'Urtarrilak', 'Otsailak', 'Martxoak', 'Apirilak', 'Maiatzak', 'Ekainak', 'Uztailak', 'Abuztuak', 'Irailak', 'Urriak', 'Azaroak', 'Abenduak']
-	month_abbreviations = ['', 'urt', 'ots', 'mar', 'api', 'mai', 'eka', 'uzt', 'abu', 'ira', 'urr', 'aza', 'abe']
-	day_names = ['', 'Asteleehna', 'Asteartea', 'Asteazkena', 'Osteguna', 'Ostirala', 'Larunbata', 'Igandea']
-	day_abbreviations = ['', 'al', 'ar', 'az', 'og', 'ol', 'lr', 'ig']
+    month_names = ['', 'xaneiro', 'febreiro', 'marzo', 'abril', 'maio', 'xuño', 'xullo', 'agosto', 'setembro', 'outubro', 'novembro', 'decembro']
+    month_abbreviations = ['', 'xan', 'feb', 'mar', 'abr', 'mai', 'xun', 'xul', 'ago', 'set', 'out', 'nov', 'dec']
+    day_names = ['', 'luns', 'martes', 'mércores', 'xoves', 'venres', 'sábado', 'domingo']
+    day_abbreviations = ['', 'lun', 'mar', 'mer', 'xov', 'ven', 'sab', 'dom']
+    ordinal_day_re = r"((?P<value>[1-3]?[0-9](?=[ºª]))[ºª])"
 

src/locales/fr/LC_MESSAGES/twblue.po

@@ -2,14 +2,14 @@ msgid ""
 msgstr ""
 "Project-Id-Version: TW Blue 0.94\n"
 "POT-Creation-Date: 2019-03-17 13:34+Hora estándar romance\n"
-"PO-Revision-Date: 2019-07-22 16:16+0200\n"
+"PO-Revision-Date: 2020-06-04 21:19+0200\n"
 "Last-Translator: Corentin BACQUÉ-CAZENAVE <corentin@progaccess33.net>\n"
-"Language-Team: Corentin BACQUÉ-CAZENAVE <corentin@progaccess.net>\n"
+"Language-Team: Oreonan <corentin@progaccess.net>\n"
 "Language: fr\n"
 "MIME-Version: 1.0\n"
 "Content-Type: text/plain; charset=UTF-8\n"
 "Content-Transfer-Encoding: 8bit\n"
-"X-Generator: Poedit 2.2.3\n"
+"X-Generator: Poedit 2.3.1\n"
 "X-Poedit-KeywordsList: _;gettext;gettext_noop\n"
 "X-Poedit-Basepath: .\n"
 "Plural-Forms: nplurals=2; plural=(n != 1);\n"
@@ -262,7 +262,7 @@ msgstr "Listes"
 #: ../src\controller\mainController.py:378
 #: ../src\controller\mainController.py:1399
 msgid "List for {}"
-msgstr "Liste de {}"
+msgstr "Liste {}"
 
 #: ../src\controller\mainController.py:381
 msgid "Searches"
@@ -1988,7 +1988,7 @@ msgstr "Téléchargement de la nouvelle version en cours..."
 
 #: ../src\update\wxUpdater.py:28
 msgid "Updating... %s of %s"
-msgstr "Mise à jour en cours... %s de %s"
+msgstr "Mise à jour en cours... %s sur %s"
 
 #: ../src\update\wxUpdater.py:31
 msgid "Done!"

src/locales/ru/LC_MESSAGES/twblue.po

@@ -6,15 +6,15 @@ msgid ""
 msgstr ""
 "Project-Id-Version: TW Blue 0.85\n"
 "POT-Creation-Date: 2019-03-17 13:34+Hora estndar romance\n"
-"PO-Revision-Date: 2018-08-15 09:47+0400\n"
-"Last-Translator: Manuel Cortez <manuel@manuelcortez.net>\n"
+"PO-Revision-Date: 2020-10-23 14:30+0300\n"
+"Last-Translator: Artem Plaksin <admin@maniyax.ru>\n"
 "Language-Team: Alexander Jaszyn <a.jaszyn@ya.ru>\n"
 "Language: ru\n"
 "MIME-Version: 1.0\n"
 "Content-Type: text/plain; charset=UTF-8\n"
 "Content-Transfer-Encoding: 8bit\n"
 "Generated-By: pygettext.py 1.5\n"
-"X-Generator: Poedit 1.5.7\n"
+"X-Generator: Poedit 1.8.8\n"
 "Plural-Forms: nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n"
 "%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);\n"
 "X-Poedit-SourceCharset: UTF-8\n"
@@ -84,24 +84,22 @@ msgid "Muted users"
 msgstr "Отключенные пользователи"
 
 #: ../src\controller\buffers\twitterBuffers.py:75
-#, fuzzy
 msgid "{username}'s timeline"
-msgstr "Открыть ленту пользователя"
+msgstr "Лента твитов {username}"
 
 #: ../src\controller\buffers\twitterBuffers.py:77
 msgid "{username}'s likes"
-msgstr ""
+msgstr "Понравившееся {username}"
 
 #: ../src\controller\buffers\twitterBuffers.py:79
 msgid "{username}'s followers"
-msgstr ""
+msgstr "Читающие {username}"
 
 #: ../src\controller\buffers\twitterBuffers.py:81
 msgid "{username}'s friends"
-msgstr ""
+msgstr "Список читаемых {username}"
 
 #: ../src\controller\buffers\twitterBuffers.py:83
-#, fuzzy
 msgid "Unknown buffer"
 msgstr "Неизвестно"
 
@@ -119,14 +117,12 @@ msgid "Write the tweet here"
 msgstr "Напишите текст твита здесь"
 
 #: ../src\controller\buffers\twitterBuffers.py:194
-#, fuzzy
 msgid "New tweet in {0}"
-msgstr "Новый твит"
+msgstr "Новый твит в {0}"
 
 #: ../src\controller\buffers\twitterBuffers.py:197
-#, fuzzy
 msgid "{0} new tweets in {1}."
-msgstr "Пользователь @{0} процитировал ваш твит: {1}"
+msgstr "@{0} процитировал ваш твит: {1}"
 
 #: ../src\controller\buffers\twitterBuffers.py:232
 #: ../src\controller\buffers\twitterBuffers.py:676
@@ -182,7 +178,7 @@ msgstr "Профиль"
 #: ../src\controller\buffers\twitterBuffers.py:634
 #: ../src\controller\buffers\twitterBuffers.py:987
 msgid "Opening item in web browser..."
-msgstr ""
+msgstr "Открыть в браузере"
 
 #: ../src\controller\buffers\twitterBuffers.py:688
 #: ../src\controller\buffers\twitterBuffers.py:855
@@ -196,12 +192,10 @@ msgid "Mention"
 msgstr "Упомянуть"
 
 #: ../src\controller\buffers\twitterBuffers.py:728
-#, fuzzy
 msgid "{0} new direct messages."
 msgstr "Новое личное сообщение"
 
 #: ../src\controller\buffers\twitterBuffers.py:731
-#, fuzzy
 msgid "This action is not supported in the buffer yet."
 msgstr "Это действие не поддерживается в данном буфере"
 
@@ -210,6 +204,8 @@ msgid ""
 "Getting more items cannot be done in this buffer. Use the direct messages "
 "buffer instead."
 msgstr ""
+"Невозможно получить больше элементов в данном буфере, используйте буфер "
+"личных сообщений вместо этого."
 
 #: ../src\controller\buffers\twitterBuffers.py:983
 #, fuzzy
@@ -379,12 +375,11 @@ msgstr "Список уже открыт"
 
 #: ../src\controller\mainController.py:1423
 #: ../src\controller\mainController.py:1439
-#, fuzzy
 msgid ""
 "An error happened while trying to connect to the server. Please try later."
 msgstr ""
-"Что-то неожиданное произошло при попытке сообщить об ошибке. Пожалуйста, "
-"повторите попытку позже"
+"При попытке подключиться к серверу произошла ошибка. Пожалуйста, попробуйте "
+"позже."
 
 #: ../src\controller\mainController.py:1475
 msgid "The auto-reading of new tweets is enabled for this buffer"
@@ -874,19 +869,19 @@ msgstr "Игнорировать"
 
 #: ../src\extra\SpellChecker\wx_ui.py:43
 msgid "I&gnore all"
-msgstr "Игнорировать все"
+msgstr "&Игнорировать все"
 
 #: ../src\extra\SpellChecker\wx_ui.py:44
 msgid "&Replace"
-msgstr "Заменить"
+msgstr "&Заменить"
 
 #: ../src\extra\SpellChecker\wx_ui.py:45
 msgid "R&eplace all"
-msgstr "Заменить все"
+msgstr "З&аменить все"
 
 #: ../src\extra\SpellChecker\wx_ui.py:46
 msgid "&Add to personal dictionary"
-msgstr "Добавить в личный словарь"
+msgstr "&Добавить в личный словарь"
 
 #: ../src\extra\SpellChecker\wx_ui.py:79
 msgid ""
@@ -1587,9 +1582,8 @@ msgid "Open URL"
 msgstr "Открыть ссылку"
 
 #: ../src\keystrokeEditor\constants.py:25
-#, fuzzy
 msgid "View in Twitter"
-msgstr "Поиск в твиттере"
+msgstr "Посмотреть в Твиттере"
 
 #: ../src\keystrokeEditor\constants.py:26
 msgid "Increase volume by 5%"
@@ -1693,7 +1687,7 @@ msgstr "Просмотр беседы"
 
 #: ../src\keystrokeEditor\constants.py:51
 msgid "Check and download updates"
-msgstr "Проверить на наличие обновлений"
+msgstr "Проверить наличие обновлений"
 
 #: ../src\keystrokeEditor\constants.py:52
 msgid ""
@@ -1816,6 +1810,9 @@ msgid ""
 "If you're sure that {0} isn't running, try deleting the file at {1}. If "
 "you're unsure of how to do this, contact the {0} developers."
 msgstr ""
+"{0} уже запущен. Закройте другой экземпляр, прежде чем запускать этот. Если "
+"вы уверены, что {0} не активен, попробуйте удалить файл по адресу {1}. Если "
+"вы не знаете, как это сделать, обратитесь к разработчикам {0}."
 
 #: ../src\sessionmanager\wxUI.py:8
 msgid "Session manager"
@@ -1920,9 +1917,8 @@ msgid "public"
 msgstr "Публичный"
 
 #: ../src\sessions\twitter\session.py:169
-#, fuzzy
 msgid "There are no more items to retrieve in this buffer."
-msgstr "Координаты отсутствуют"
+msgstr "В этом буфере больше нет элементов для извлечения."
 
 #: ../src\sessions\twitter\session.py:215
 msgid "%s failed.  Reason: %s"
@@ -2048,7 +2044,7 @@ msgstr "Тренды"
 
 #: ../src\wxUI\buffers\trends.py:18
 msgid "Tweet about this trend"
-msgstr "Tweet о тренде"
+msgstr "Твитнуть о тренде"
 
 #: ../src\wxUI\buffers\trends.py:19 ../src\wxUI\menus.py:96
 msgid "Search topic"
@@ -2265,6 +2261,8 @@ msgid ""
 "{0} quit unexpectedly the last time it was run. If the problem persists, "
 "please report it to the {0} developers."
 msgstr ""
+"{0} неожиданно закончил свою работу при последнем запуске. Если проблема не "
+"устранена, пожалуйста, сообщите об этом разработчикам {0}."
 
 #: ../src\wxUI\dialogs\attach.py:9
 msgid "Add an attachment"
@@ -2362,6 +2360,7 @@ msgstr "Использование удлинителя твитов (может
 #: ../src\wxUI\dialogs\configuration.py:48
 msgid "Remember state for mention all and long tweet"
 msgstr ""
+"Запоминать состояние для упоминаний всех пользователей и длинных твитов"
 
 #: ../src\wxUI\dialogs\configuration.py:51
 msgid "Keymap"
@@ -2413,7 +2412,7 @@ msgstr ""
 
 #: ../src\wxUI\dialogs\configuration.py:116
 msgid "Retweet mode"
-msgstr "Стиль ретвита"
+msgstr "Режим ретвита"
 
 #: ../src\wxUI\dialogs\configuration.py:122
 msgid "Show screen names instead of full names"
@@ -2429,11 +2428,11 @@ msgstr ""
 
 #: ../src\wxUI\dialogs\configuration.py:134
 msgid "Enable automatic speech feedback"
-msgstr ""
+msgstr "Включить автоматический речевой вывод"
 
 #: ../src\wxUI\dialogs\configuration.py:136
 msgid "Enable automatic Braille feedback"
-msgstr ""
+msgstr "Включить автоматический брайлевский вывод"
 
 #: ../src\wxUI\dialogs\configuration.py:144
 msgid "Status"
@@ -2515,7 +2514,7 @@ msgstr "Устройство записи"
 
 #: ../src\wxUI\dialogs\configuration.py:299
 msgid "Sound pack"
-msgstr "Пакет звуков"
+msgstr "Звуковая схема"
 
 #: ../src\wxUI\dialogs\configuration.py:305
 msgid "Indicate audio tweets with sound"
@@ -2539,7 +2538,7 @@ msgstr "API ключ SndUp"
 
 #: ../src\wxUI\dialogs\configuration.py:353
 msgid "{0} preferences"
-msgstr "Настройки {0}"
+msgstr "Параметры {0}"
 
 #: ../src\wxUI\dialogs\configuration.py:364
 msgid "Proxy"
@@ -2819,9 +2818,8 @@ msgid "Source: "
 msgstr "Клиент"
 
 #: ../src\wxUI\dialogs\message.py:342 ../src\wxUI\dialogs\message.py:420
-#, fuzzy
 msgid "Date: "
-msgstr "Дата"
+msgstr "Дата: "
 
 #: ../src\wxUI\dialogs\message.py:405
 msgid "View"
@@ -2890,7 +2888,7 @@ msgstr "Детали"
 
 #: ../src\wxUI\dialogs\show_user.py:16
 msgid "&Go to URL"
-msgstr "Перейти по ссылке"
+msgstr "&Перейти по ссылке"
 
 #: ../src\wxUI\dialogs\trends.py:12
 msgid "View trending topics"
@@ -3031,12 +3029,11 @@ msgstr "н&е нравится"
 
 #: ../src\wxUI\menus.py:15 ../src\wxUI\menus.py:35 ../src\wxUI\menus.py:51
 msgid "&Open URL"
-msgstr "Открыть  ссылку"
+msgstr "&Открыть  ссылку"
 
 #: ../src\wxUI\menus.py:17 ../src\wxUI\menus.py:53 ../src\wxUI\menus.py:86
-#, fuzzy
 msgid "&Open in Twitter"
-msgstr "Поиск в твиттере"
+msgstr "Открыть в &Твиттере"
 
 #: ../src\wxUI\menus.py:19 ../src\wxUI\menus.py:37 ../src\wxUI\menus.py:55
 msgid "&Play audio"
@@ -3246,7 +3243,7 @@ msgstr "Сайт {0}"
 
 #: ../src\wxUI\view.py:77
 msgid "Get soundpacks for TWBlue"
-msgstr ""
+msgstr "Получить звуковые схемы для TWBlue"
 
 #: ../src\wxUI\view.py:78
 msgid "About &{0}"

src/locales/sr/LC_MESSAGES/twblue.po

@@ -5,16 +5,16 @@
 msgid ""
 msgstr ""
 "Project-Id-Version: TwBlue 0.80\n"
-"POT-Creation-Date: 2019-03-17 13:34+Hora estndar romance\n"
-"PO-Revision-Date: 2018-08-07 13:46-0500\n"
-"Last-Translator: Manuel Cortez <manuel@manuelcortez.net>\n"
+"POT-Creation-Date: 2019-03-17 13:34+Hora estándar romance\n"
+"PO-Revision-Date: 2021-06-25 18:32+0100\n"
+"Last-Translator: Nikola Jović <wwenikola123@gmail.com>\n"
 "Language-Team: Aleksandar Đurić <agasoft@gmail.com>\n"
 "Language: sr_RS@latin\n"
 "MIME-Version: 1.0\n"
 "Content-Type: text/plain; charset=UTF-8\n"
 "Content-Transfer-Encoding: 8bit\n"
 "Generated-By: pygettext.py 1.5\n"
-"X-Generator: Poedit 2.0.1\n"
+"X-Generator: Poedit 1.6.10\n"
 "X-Poedit-Bookmarks: -1,442,-1,-1,-1,-1,-1,-1,-1,-1\n"
 "X-Poedit-SourceCharset: UTF-8\n"
 
@@ -82,26 +82,24 @@ msgid "Muted users"
 msgstr "Utišani korisnici"
 
 #: ../src\controller\buffers\twitterBuffers.py:75
-#, fuzzy
 msgid "{username}'s timeline"
-msgstr "Otvori korisničku vremensku crtu"
+msgstr "Vremenska linija korisnika {username}"
 
 #: ../src\controller\buffers\twitterBuffers.py:77
 msgid "{username}'s likes"
-msgstr ""
+msgstr "Sviđanja korisnika {username}"
 
 #: ../src\controller\buffers\twitterBuffers.py:79
 msgid "{username}'s followers"
-msgstr ""
+msgstr "Pratioci korisnika {username}"
 
 #: ../src\controller\buffers\twitterBuffers.py:81
 msgid "{username}'s friends"
-msgstr ""
+msgstr "Prijatelji korisnika {username}"
 
 #: ../src\controller\buffers\twitterBuffers.py:83
-#, fuzzy
 msgid "Unknown buffer"
-msgstr "Nepoznato"
+msgstr "Nepoznat kanal"
 
 #: ../src\controller\buffers\twitterBuffers.py:86
 #: ../src\controller\buffers\twitterBuffers.py:1242
@@ -117,14 +115,12 @@ msgid "Write the tweet here"
 msgstr "Otkucajte tvit ovde:"
 
 #: ../src\controller\buffers\twitterBuffers.py:194
-#, fuzzy
 msgid "New tweet in {0}"
-msgstr "Novi tvit"
+msgstr "Novi tvit u kanalu {0}"
 
 #: ../src\controller\buffers\twitterBuffers.py:197
-#, fuzzy
 msgid "{0} new tweets in {1}."
-msgstr "@{0} citira vaš tvit: {1}"
+msgstr "{0} novih tvitova u kanalu {1}."
 
 #: ../src\controller\buffers\twitterBuffers.py:232
 #: ../src\controller\buffers\twitterBuffers.py:676
@@ -180,7 +176,7 @@ msgstr "Podaci o korisniku"
 #: ../src\controller\buffers\twitterBuffers.py:634
 #: ../src\controller\buffers\twitterBuffers.py:987
 msgid "Opening item in web browser..."
-msgstr ""
+msgstr "Otvaram stavku u Web pretraživaču..."
 
 #: ../src\controller\buffers\twitterBuffers.py:688
 #: ../src\controller\buffers\twitterBuffers.py:855
@@ -194,30 +190,28 @@ msgid "Mention"
 msgstr "Spomeni"
 
 #: ../src\controller\buffers\twitterBuffers.py:728
-#, fuzzy
 msgid "{0} new direct messages."
-msgstr "Nova direktna poruka"
+msgstr "{0} novih direktnih poruka."
 
 #: ../src\controller\buffers\twitterBuffers.py:731
-#, fuzzy
 msgid "This action is not supported in the buffer yet."
-msgstr "Ova radnja nije podržana na ovom kanalu"
+msgstr "Ova radnja još uvek nije podržana na ovom kanalu."
 
 #: ../src\controller\buffers\twitterBuffers.py:741
 msgid ""
 "Getting more items cannot be done in this buffer. Use the direct messages "
 "buffer instead."
 msgstr ""
+"Nemoguće preuzeti dodatne stavke za ovaj kanal. Umesto toga, koristite kanal "
+"direktne poruke."
 
 #: ../src\controller\buffers\twitterBuffers.py:983
-#, fuzzy
 msgid "{0} new followers."
-msgstr "Novi pratilac."
+msgstr "{0} novih pratilaca."
 
 #: ../src\controller\buffers\twitterBuffers.py:1266
-#, fuzzy
 msgid "This action is not supported in the buffer, yet."
-msgstr "Ova radnja nije podržana na ovom kanalu"
+msgstr "Ova radnja još uvek nije podržana na ovom kanalu"
 
 #: ../src\controller\mainController.py:273
 msgid "Ready"
@@ -308,9 +302,8 @@ msgid "{0} not found."
 msgstr "{0} nije pronađen."
 
 #: ../src\controller\mainController.py:482
-#, fuzzy
 msgid "Filters cannot be applied on this buffer"
-msgstr "Ova radnja nije podržana na ovom kanalu"
+msgstr "Filteri se ne mogu primeniti na ovaj kanal"
 
 #: ../src\controller\mainController.py:535
 #: ../src\controller\mainController.py:552
@@ -319,9 +312,8 @@ msgid "Select the user"
 msgstr "Izaberite korisnika"
 
 #: ../src\controller\mainController.py:809 ../src\controller\messages.py:236
-#, fuzzy
 msgid "MMM D, YYYY. H:m"
-msgstr "dddd, MMMM D, YYYY H:m:s"
+msgstr "MMM D, YYYY. H:m"
 
 #: ../src\controller\mainController.py:934
 msgid "Conversation with {0}"
@@ -379,12 +371,11 @@ msgstr "Ova lista je već otvorena"
 
 #: ../src\controller\mainController.py:1423
 #: ../src\controller\mainController.py:1439
-#, fuzzy
 msgid ""
 "An error happened while trying to connect to the server. Please try later."
 msgstr ""
-"Dogodilo se nešto neočekivano prilikom prijavljivanja greške. Molimo vas da "
-"pokušate kasnije."
+"Došlo je do greške pri pokušaju povezivanja na server. Molimo pokušajte "
+"kasnije."
 
 #: ../src\controller\mainController.py:1475
 msgid "The auto-reading of new tweets is enabled for this buffer"
@@ -475,9 +466,8 @@ msgid "%s - %s characters"
 msgstr "%s - %s znakova"
 
 #: ../src\controller\messages.py:262
-#, fuzzy
 msgid "View item"
-msgstr "Vidi liste"
+msgstr "Prikaži stavku"
 
 #: ../src\controller\settings.py:75
 msgid "Direct connection"
@@ -533,7 +523,8 @@ msgstr "Korisnik je suspendovan."
 msgid "Information for %s"
 msgstr "Informacija za %s"
 
-#: ../src\controller\user.py:66 ../src\extra\AudioUploader\audioUploader.py:124
+#: ../src\controller\user.py:66
+#: ../src\extra\AudioUploader\audioUploader.py:124
 msgid "Discarded"
 msgstr "Odbačeno"
 
@@ -658,9 +649,8 @@ msgstr "Snimam zvučni zapis..."
 
 #: ../src\extra\AudioUploader\transfer.py:78
 #: ../src\extra\AudioUploader\transfer.py:84
-#, fuzzy
 msgid "Error in file upload: {0}"
-msgstr "Kod greške {0}"
+msgstr "Greška u otpremanju datoteke: {0}"
 
 #: ../src\extra\AudioUploader\utils.py:27 ../src\update\utils.py:27
 msgid "%d day, "
@@ -871,28 +861,24 @@ msgid "Suggestions"
 msgstr "Predlozi"
 
 #: ../src\extra\SpellChecker\wx_ui.py:42
-#, fuzzy
 msgid "&Ignore"
-msgstr "Zamenari."
+msgstr "&Zanemari"
 
 #: ../src\extra\SpellChecker\wx_ui.py:43
-#, fuzzy
 msgid "I&gnore all"
-msgstr "Zanemari sve"
+msgstr "Z&anemai sve"
 
 #: ../src\extra\SpellChecker\wx_ui.py:44
-#, fuzzy
 msgid "&Replace"
-msgstr "Zameni"
+msgstr "&Zameni"
 
 #: ../src\extra\SpellChecker\wx_ui.py:45
-#, fuzzy
 msgid "R&eplace all"
-msgstr "Zameni sve"
+msgstr "Zam&eni sve"
 
 #: ../src\extra\SpellChecker\wx_ui.py:46
 msgid "&Add to personal dictionary"
-msgstr ""
+msgstr "&Dodaj u lični rečnik"
 
 #: ../src\extra\SpellChecker\wx_ui.py:79
 msgid ""
@@ -1553,9 +1539,8 @@ msgid "Like a tweet"
 msgstr "Označi tvit sa sviđa mi se"
 
 #: ../src\keystrokeEditor\constants.py:15
-#, fuzzy
 msgid "Like/unlike a tweet"
-msgstr "Označi tvit sa ne sviđa mi se"
+msgstr "Označi da ti se tvit sviđa / ne sviđa"
 
 #: ../src\keystrokeEditor\constants.py:16
 msgid "Unlike a tweet"
@@ -1594,9 +1579,8 @@ msgid "Open URL"
 msgstr "Otvori vezu"
 
 #: ../src\keystrokeEditor\constants.py:25
-#, fuzzy
 msgid "View in Twitter"
-msgstr "Pretraži ttwitter"
+msgstr "Prikaži na Twitteru"
 
 #: ../src\keystrokeEditor\constants.py:26
 msgid "Increase volume by 5%"
@@ -1715,9 +1699,8 @@ msgid "Opens the global settings dialogue"
 msgstr "Otvara dijalog globalnih podešavanja"
 
 #: ../src\keystrokeEditor\constants.py:54
-#, fuzzy
 msgid "Opens the list manager"
-msgstr "Upravljanje listama"
+msgstr "Otvara upravljanje listama"
 
 #: ../src\keystrokeEditor\constants.py:55
 msgid "Opens the account settings dialogue"
@@ -1821,6 +1804,10 @@ msgid ""
 "If you're sure that {0} isn't running, try deleting the file at {1}. If "
 "you're unsure of how to do this, contact the {0} developers."
 msgstr ""
+"{0} je  već pokrenut. Zatvorite druge pokrenute prozore aplikacije pre "
+"pokretanja ovog. Ako ste sigurni da {0} nije pokrenut, pokušajte da obrišete "
+"datoteku koja se nalazi na {1}. Ako niste sigurni kako ovo da uradite, "
+"kontaktirajte {0} programere."
 
 #: ../src\sessionmanager\wxUI.py:8
 msgid "Session manager"
@@ -1926,9 +1913,8 @@ msgid "public"
 msgstr "Javno"
 
 #: ../src\sessions\twitter\session.py:169
-#, fuzzy
 msgid "There are no more items to retrieve in this buffer."
-msgstr "Nema koordinata u ovom tvitu."
+msgstr "Nema više stavki koje se mogu preuzeti u ovom kanalu."
 
 #: ../src\sessions\twitter\session.py:215
 msgid "%s failed.  Reason: %s"
@@ -1951,13 +1937,12 @@ msgid "Error code {0}"
 msgstr "Kod greške {0}"
 
 #: ../src\sessions\twitter\wxUI.py:6
-#, fuzzy
 msgid "Authorising account..."
-msgstr "Autorizovan nalog %d"
+msgstr "Autorizacija naloga..."
 
 #: ../src\sessions\twitter\wxUI.py:9
 msgid "Enter your PIN code here"
-msgstr ""
+msgstr "Ovde upišite vaš PIN kod"
 
 #: ../src\sound.py:159
 msgid "Stopped."
@@ -2266,19 +2251,20 @@ msgstr ""
 "suspendovan sa tvitera."
 
 #: ../src\wxUI\commonMessageDialogs.py:85
-#, fuzzy
 msgid "Do you really want to delete this filter?"
-msgstr "Želite li zaista da izbrišete ovu listu?"
+msgstr "Da li zaista želite da izbrišete ovaj filter?"
 
 #: ../src\wxUI\commonMessageDialogs.py:88
 msgid "This filter already exists. Please use a different title"
-msgstr ""
+msgstr "Ovaj filter već postoji. Molimo koristite drugi naziv"
 
 #: ../src\wxUI\commonMessageDialogs.py:94
 msgid ""
 "{0} quit unexpectedly the last time it was run. If the problem persists, "
 "please report it to the {0} developers."
 msgstr ""
+"{0} je neočekivano zatvoren pri poslednjem pokretanju. Ako se problem "
+"nastavi, molimo prijavite ga{0} programerima."
 
 #: ../src\wxUI\dialogs\attach.py:9
 msgid "Add an attachment"
@@ -2343,11 +2329,11 @@ msgstr "Pitaj pre zatvaranja {0}"
 
 #: ../src\wxUI\dialogs\configuration.py:27
 msgid "Disable Streaming functions"
-msgstr ""
+msgstr "Onemogući streaming funkcije"
 
 #: ../src\wxUI\dialogs\configuration.py:30
 msgid "Buffer update interval, in minutes"
-msgstr ""
+msgstr "Interval ažuriranja kanala, u minutima"
 
 #: ../src\wxUI\dialogs\configuration.py:36
 msgid "Play a sound when {0} launches"
@@ -2378,7 +2364,7 @@ msgstr ""
 
 #: ../src\wxUI\dialogs\configuration.py:48
 msgid "Remember state for mention all and long tweet"
-msgstr ""
+msgstr "Pamti stanje opcija spomeni sve i dug tvit"
 
 #: ../src\wxUI\dialogs\configuration.py:51
 msgid "Keymap"
@@ -2446,11 +2432,11 @@ msgstr ""
 
 #: ../src\wxUI\dialogs\configuration.py:134
 msgid "Enable automatic speech feedback"
-msgstr ""
+msgstr "Omogući automatske govorne povratne informacije"
 
 #: ../src\wxUI\dialogs\configuration.py:136
 msgid "Enable automatic Braille feedback"
-msgstr ""
+msgstr "Omogući automatske povratne informacije na brajevom redu"
 
 #: ../src\wxUI\dialogs\configuration.py:144
 msgid "Status"
@@ -2564,7 +2550,7 @@ msgstr "Proksi"
 
 #: ../src\wxUI\dialogs\configuration.py:373
 msgid "Feedback"
-msgstr ""
+msgstr "Povratne informacije"
 
 #: ../src\wxUI\dialogs\configuration.py:377
 msgid "Buffers"
@@ -2583,81 +2569,74 @@ msgid "Save"
 msgstr "Sačuvaj"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:15
-#, fuzzy
 msgid "Create a filter  for this buffer"
-msgstr "Stvori kanal trenutnih tema u trendu"
+msgstr "Napravi filter za ovaj kanal"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:16
 msgid "Filter title"
-msgstr ""
+msgstr "Naziv filtera"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:25
 #: ../src\wxUI\dialogs\filterDialogs.py:125
 msgid "Filter by word"
-msgstr ""
+msgstr "Filtriraj na osnovu reči"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:26
 msgid "Ignore tweets wich contain the following word"
-msgstr ""
+msgstr "Zanemari tvitove koji sadrže sledeću reč"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:27
-#, fuzzy
 msgid "Ignore tweets without the following word"
-msgstr "Zanemari tvitove iz ovog klijenta"
+msgstr "Zanemari tvitove bez sledeće reči"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:32
 msgid "word"
-msgstr ""
+msgstr "Reč"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:37
-#, fuzzy
 msgid "Allow retweets"
-msgstr "Prikaži tvit"
+msgstr "Dozvoli retvitove"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:38
 msgid "Allow quoted tweets"
-msgstr ""
+msgstr "Dozvoli citirane tvitove"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:39
-#, fuzzy
 msgid "Allow replies"
-msgstr "Vremenska linija pratilaca"
+msgstr "Dozvoli odgovore"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:47
 msgid "Use this term as a regular expression"
-msgstr ""
+msgstr "Koristi ovaj termin kao regulara nizraz"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:49
 #: ../src\wxUI\dialogs\filterDialogs.py:125
-#, fuzzy
 msgid "Filter by language"
-msgstr "Izvorni jezik"
+msgstr "Filtriraj na osnovu jezika"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:50
 msgid "Load tweets in the following languages"
-msgstr ""
+msgstr "Učitaj tvitove na sledećim jezicima"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:51
 msgid "Ignore tweets in the following languages"
-msgstr ""
+msgstr "Zanemari tvitove na sledećim jezicima"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:52
 msgid "Don't filter by language"
-msgstr ""
+msgstr "Ne filtriraj na osnovu jezika"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:63
-#, fuzzy
 msgid "Supported languages"
-msgstr "Izvorni jezik"
+msgstr "Podržani jezici"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:68
 msgid "Add selected language to filter"
-msgstr ""
+msgstr "Dodaj izabrani jezik u filter"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:72
-#, fuzzy
 msgid "Selected languages"
-msgstr "Izvorni jezik"
+msgstr "Izabrani jezici"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:74
 #: ../src\wxUI\dialogs\filterDialogs.py:132 ../src\wxUI\dialogs\lists.py:20
@@ -2666,17 +2645,16 @@ msgid "Remove"
 msgstr "Ukloni"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:122
-#, fuzzy
 msgid "Manage filters"
-msgstr "Upravljaj nalozima"
+msgstr "Upravljanje filterima"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:124
 msgid "Filters"
-msgstr ""
+msgstr "Filteri"
 
 #: ../src\wxUI\dialogs\filterDialogs.py:125
 msgid "Filter"
-msgstr ""
+msgstr "Filter"
 
 #: ../src\wxUI\dialogs\find.py:12
 msgid "Find in current buffer"
@@ -2844,9 +2822,8 @@ msgid "Source: "
 msgstr "Izvor:"
 
 #: ../src\wxUI\dialogs\message.py:342 ../src\wxUI\dialogs\message.py:420
-#, fuzzy
 msgid "Date: "
-msgstr "Datum"
+msgstr "Datum:"
 
 #: ../src\wxUI\dialogs\message.py:405
 msgid "View"
@@ -2942,9 +2919,8 @@ msgid "Update your profile"
 msgstr "Ažurirajte vaš profil"
 
 #: ../src\wxUI\dialogs\update_profile.py:11
-#, fuzzy
 msgid "&Name (50 characters maximum)"
-msgstr "Ime, najviše 20 znakova"
+msgstr "&Ime(najviše 50 znakova)"
 
 #: ../src\wxUI\dialogs\update_profile.py:22
 msgid "&Website"
@@ -3060,9 +3036,8 @@ msgid "&Open URL"
 msgstr "otvori vezu"
 
 #: ../src\wxUI\menus.py:17 ../src\wxUI\menus.py:53 ../src\wxUI\menus.py:86
-#, fuzzy
 msgid "&Open in Twitter"
-msgstr "Pretraži ttwitter"
+msgstr "&Otvori na Twitteru"
 
 #: ../src\wxUI\menus.py:19 ../src\wxUI\menus.py:37 ../src\wxUI\menus.py:55
 msgid "&Play audio"
@@ -3215,14 +3190,12 @@ msgid "New &trending topics buffer..."
 msgstr "Novi kanal sa temama u trendu"
 
 #: ../src\wxUI\view.py:54
-#, fuzzy
 msgid "Create a &filter"
-msgstr "Stvori novu listu"
+msgstr "Napravi &filter"
 
 #: ../src\wxUI\view.py:55
-#, fuzzy
 msgid "&Manage filters"
-msgstr "Upravljaj nalozima"
+msgstr "&Upravljanje filterima"
 
 #: ../src\wxUI\view.py:56
 msgid "Find a string in the currently focused buffer..."
@@ -3274,7 +3247,7 @@ msgstr "{0} &website"
 
 #: ../src\wxUI\view.py:77
 msgid "Get soundpacks for TWBlue"
-msgstr ""
+msgstr "Nabavi zvučne pakete za TW Blue"
 
 #: ../src\wxUI\view.py:78
 msgid "About &{0}"

src/main.py

@@ -65,6 +65,7 @@ log = logging.getLogger("main")
 def setup():
     log.debug("Starting " + application.name + " %s" % (application.version,))
     config.setup()
+    proxy_setup()
     log.debug("Using %s %s" % (platform.system(), platform.architecture()[0]))
     log.debug("Application path is %s" % (paths.app_path(),))
     log.debug("config path  is %s" % (paths.config_path(),))
@@ -77,6 +78,7 @@ def setup():
     from controller import mainController
     from sessionmanager import sessionManager
     app = widgetUtils.mainLoopObject()
+    check_pid()
     if system == "Windows":
         if config.app["app-settings"]["donation_dialog_displayed"] == False:
             donation()
@@ -90,7 +92,6 @@ def setup():
     if hasattr(sm.view, "destroy"):
         sm.view.destroy()
     del sm
- check_pid()
     r = mainController.Controller()
     r.view.show()
     r.do_work()
@@ -101,6 +102,18 @@ def setup():
         GLib.idle_add(r.start)
     app.run()
 
+def proxy_setup():
+    if config.app["proxy"]["server"] != "" and config.app["proxy"]["type"] > 0:
+        log.debug("Loading proxy settings")
+        proxy_url = config.app["proxy"]["server"] + ":" + str(config.app["proxy"]["port"])
+        if config.app["proxy"]["user"] != "" and config.app["proxy"]["password"] != "":
+            proxy_url = config.app["proxy"]["user"] + ":" + config.app["proxy"]["password"] + "@" + proxy_url
+        elif config.app["proxy"]["user"] != "" and config.proxyTypes[config.app["proxy"]["type"]] in ["socks4", "socks4a"]:
+            proxy_url = config.app["proxy"]["user"] + "@" + proxy_url
+        proxy_url = config.proxyTypes[config.app["proxy"]["type"]] + "://" + proxy_url
+        os.environ["HTTP_PROXY"] = proxy_url
+        os.environ["HTTPS_PROXY"] = proxy_url
+
 def donation():
     dlg = commonMessageDialogs.donation()
     if dlg == widgetUtils.YES:

src/mysc/restart.py

@@ -1,6 +1,7 @@
 # -*- coding: cp1252
 from __future__ import unicode_literals
 import sys, os
+import application
 
 def restart_program():
     """ Function that restarts the application if is executed."""
@@ -9,4 +10,7 @@ def restart_program():
         args.insert(0, sys.executable)
     if sys.platform == 'win32':
         args = ['"%s"' % arg for arg in args]
+    pidpath = os.path.join(os.getenv("temp"), "{}.pid".format(application.name))
+    if os.path.exists(pidpath):
+        os.remove(pidpath)
     os.execv(sys.executable, args)

src/mysc/thread_utils.py

@@ -1,21 +1,15 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
-from future import standard_library
-standard_library.install_aliases()
 import logging
 log = logging.getLogger("mysc.thread_utils")
 import threading
 import wx
 from pubsub import pub
-from twython import TwythonRateLimitError
 
 def call_threaded(func, *args, **kwargs):
     #Call the given function in a daemonized thread and return the thread.
     def new_func(*a, **k):
         try:
             func(*a, **k)
-  except TwythonRateLimitError:
-   pass
         except:
             log.exception("Thread %d with function %r, args of %r, and kwargs of %r failed to run." % (threading.current_thread().ident, func, a, k))
 #   pass
@@ -23,17 +17,3 @@ def call_threaded(func, *args, **kwargs):
     thread.daemon = True
     thread.start()
     return thread
-
-def stream_threaded(func, *args, **kwargs):
- def new_func(*a, **k):
-  try:
-   func(**k)
-  except Exception as msg:
-   log.error("Error in stream with args: %r" % (a,))
-   log.error(msg.message)
-   pub.sendMessage("stream-error", session=a[0])
-
- thread = threading.Thread(target=new_func, args=args, kwargs=kwargs)
- thread.daemon = True
- thread.start()
- return thread

src/run_tests.py

@@ -0,0 +1,18 @@
+# -*- coding: utf-8 -*-
+import unittest
+
+testmodules = ["test.test_cache"]
+
+suite = unittest.TestSuite()
+
+for t in testmodules:
+    try:
+        # If the module defines a suite() function, call it to get the suite.
+        mod = __import__(t, globals(), locals(), ['suite'])
+        suitefn = getattr(mod, 'suite')
+        suite.addTest(suitefn())
+    except (ImportError, AttributeError):
+        # else, just load all the test cases from the module.
+        suite.addTest(unittest.defaultTestLoader.loadTestsFromName(t))
+
+unittest.TextTestRunner(verbosity=2).run(suite)

src/sessionmanager/sessionManager.py

@@ -1,8 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import absolute_import
-from __future__ import unicode_literals
-from builtins import str
-from builtins import object
 import shutil
 import widgetUtils
 import platform
@@ -21,7 +17,7 @@ from sessions.twitter import session
 from . import manager
 import config_utils
 import config
-
+from tweepy.error import TweepError
 log = logging.getLogger("sessionmanager.sessionManager")
 
 class sessionManagerController(object):
@@ -85,11 +81,18 @@ class sessionManagerController(object):
             s = session.Session(i)
             s.get_configuration()
             if i not in config.app["sessions"]["ignored_sessions"]:
+                try:
                     s.login()
+                except TweepError:
+                    self.show_auth_error(s.settings["twitter"]["user_name"])
+                    continue
             sessions.sessions[i] = s
             self.new_sessions[i] = s
 #  self.view.destroy()
 
+    def show_auth_error(self, user_name):
+        error = view.auth_error(user_name)
+
     def manage_new_account(self, *args, **kwargs):
         if self.view.new_account_dialog() == widgetUtils.YES:
             location = (str(time.time())[-6:])

src/sessionmanager/wxUI.py

@@ -76,3 +76,6 @@ class sessionManagerWindow(wx.Dialog):
 
     def destroy(self):
         self.Destroy()
+
+def auth_error(user_name):
+    return wx.MessageDialog(None, _("TWBlue is unable to authenticate the account for {} in Twitter. It might be due to an invalid or expired token, revoqued access to the application, or after an account reactivation. Please remove the account manually from your Twitter sessions in order to stop seeing this message.").format(user_name,), _("Authentication error for session {}").format(user_name,), wx.OK).ShowModal()

src/sessions/base.py

@@ -1,9 +1,5 @@
 # -*- coding: utf-8 -*-
 """ A base class to be derived in possible new sessions for TWBlue and services."""
-from __future__ import absolute_import
-from __future__ import unicode_literals
-from builtins import str
-from builtins import object
 import os
 import paths
 import output
@@ -11,9 +7,8 @@ import time
 import sound
 import logging
 import config_utils
-import shelve
+import sqlitedict
 import application
-import os
 from . import session_exceptions as Exceptions
 log = logging.getLogger("sessionmanager.session")
 
@@ -59,7 +54,7 @@ class baseSession(object):
         log.debug("Creating config file %s" % (file_,))
         self.settings = config_utils.load_config(os.path.join(paths.config_path(), file_), os.path.join(paths.app_path(), "Conf.defaults"))
         self.init_sound()
-		self.deshelve()
+        self.load_persistent_data()
 
     def init_sound(self):
         try: self.sound = sound.soundSystem(self.settings["sound"])
@@ -73,48 +68,88 @@ class baseSession(object):
     def authorise(self):
         pass
 
-	def shelve(self):
-		"""Shelve the database to allow for persistance."""
-		shelfname=os.path.join(paths.config_path(), str(self.session_id)+"/cache")
-		if self.settings["general"]["persist_size"] == 0:
-			if os.path.exists(shelfname+".dat"):
-				os.remove(shelfname+".dat")
-			return
-		try:
-			if not os.path.exists(shelfname+".dat"):
-				output.speak("Generating database, this might take a while.",True)
-			shelf=shelve.open(os.path.join(paths.config_path(), shelfname),'c')
-			for key,value in list(self.db.items()):
-				if type(key) != str and type(key) != str:
-					output.speak("Uh oh, while shelving the database, a key of type " + str(type(key)) + " has been found. It will be converted to type str, but this will cause all sorts of problems on deshelve. Please bring this to the attention of the " + application.name + " developers immediately. More information about the error will be written to the error log.",True)
-					log.error("Uh oh, " + str(key) + " is of type " + str(type(key)) + "!")
-				if type(value) == list and self.settings["general"]["persist_size"] != -1 and len(value) > self.settings["general"]["persist_size"]:
-					shelf[key]=value[self.settings["general"]["persist_size"]:]
+    def get_sized_buffer(self, buffer, size, reversed=False):
+        """ Returns a list with the amount of items specified by size."""
+        if isinstance(buffer, list) and size != -1 and len(buffer) > size:
+            log.debug("Requesting {} items from a list of {} items. Reversed mode: {}".format(size, len(buffer), reversed))
+            if reversed == True:
+                return buffer[:size]
             else:
-					shelf[key]=value
-			shelf.close()
-		except:
-			output.speak("An exception occurred while shelving the " + application.name + " database. It will be deleted and rebuilt automatically. If this error persists, send the error log to the " + application.name + " developers.",True)
-			log.exception("Exception while shelving" + shelfname)
-			os.remove(shelfname)
+                return buffer[len(buffer)-size:]
+        else:
+            return buffer
 
-	def deshelve(self):
-		"""Import a shelved database."""
-		shelfname=os.path.join(paths.config_path(), str(self.session_id)+"/cache")
+    def save_persistent_data(self):
+        """ Save the data to a persistent sqlite backed file. ."""
+        dbname=os.path.join(paths.config_path(), str(self.session_id), "cache.db")
+        log.debug("Saving storage information...")
+        # persist_size set to 0 means not saving data actually.
         if self.settings["general"]["persist_size"] == 0:
-			if os.path.exists(shelfname+".dat"):
-				os.remove(shelfname+".dat")
+            if os.path.exists(dbname):
+                os.remove(dbname)
             return
+        # Let's check if we need to create a new SqliteDict object (when loading db in memory) or we just need to call to commit in self (if reading from disk).db.
+        # If we read from disk, we cannot modify the buffer size here as we could damage the app's integrity.
+        # We will modify buffer's size (managed by persist_size) upon loading the db into memory in app startup.
+        if self.settings["general"]["load_cache_in_memory"] and isinstance(self.db, dict):
+            log.debug("Opening database to dump memory contents...")
+            db=sqlitedict.SqliteDict(dbname, 'c')
+            for k in self.db.keys():
+                sized_buff = self.get_sized_buffer(self.db[k], self.settings["general"]["persist_size"], self.settings["general"]["reverse_timelines"])
+                db[k] = sized_buff
+            db.commit(blocking=True)
+            db.close()
+            log.debug("Data has been saved in the database.")
+        else:
             try:
-			shelf=shelve.open(os.path.join(paths.config_path(), shelfname),'c')
-			for key,value in list(shelf.items()):
-				self.db[key]=value
-			shelf.close()
+                log.debug("Syncing new data to disk...")
+                if hasattr(self.db, "commit"):
+                    self.db.commit()
             except:
-			output.speak("An exception occurred while deshelving the " + application.name + " database. It will be deleted and rebuilt automatically. If this error persists, send the error log to the " + application.name + " developers.",True)
-			log.exception("Exception while deshelving" + shelfname)
+                output.speak(_("An exception occurred while saving the {app} database. It will be deleted and rebuilt automatically. If this error persists, send the error log to the {app} developers.").format(app=application.name),True)
+                log.exception("Exception while saving {}".format(dbname))
+                os.remove(dbname)
+
+    def load_persistent_data(self):
+        """Import data from a database file from user config."""
+        log.debug("Loading storage data...")
+        dbname=os.path.join(paths.config_path(), str(self.session_id), "cache.db")
+        # If persist_size is set to 0, we should remove the db file as we are no longer going to save anything.
+        if self.settings["general"]["persist_size"] == 0:
+            if os.path.exists(dbname):
+                os.remove(dbname)
+            # Let's return from here, as we are not loading anything.
+            return
+        # try to load the db file.
         try:
-				os.remove(shelfname)
+            log.debug("Opening database...")
+            db=sqlitedict.SqliteDict(os.path.join(paths.config_path(), dbname), 'c')
+            # If load_cache_in_memory is set to true, we will load the whole database into memory for faster access.
+            # This is going to be faster when retrieving specific objects, at the cost of more memory.
+            # Setting this to False will read the objects from database as they are needed, which might be slower for bigger datasets.
+            if self.settings["general"]["load_cache_in_memory"]:
+                log.debug("Loading database contents into memory...")
+                for k in db.keys():
+                    self.db[k] = db[k]
+                db.commit(blocking=True)
+                db.close()
+                log.debug("Contents were loaded successfully.")
+            else:
+                log.debug("Instantiating database from disk.")
+                self.db = db
+                # We must make sure we won't load more than the amount of buffer specified.
+                log.debug("Checking if we will load all content...")
+                for k in self.db.keys():
+                    sized_buffer = self.get_sized_buffer(self.db[k], self.settings["general"]["persist_size"], self.settings["general"]["reverse_timelines"])
+                    self.db[k] = sized_buffer
+            if self.db.get("cursors") == None:
+                cursors = dict(direct_messages=-1)
+                self.db["cursors"] = cursors
+        except:
+            output.speak(_("An exception occurred while loading the {app} database. It will be deleted and rebuilt automatically. If this error persists, send the error log to the {app} developers.").format(app=application.name), True)
+            log.exception("Exception while loading {}".format(dbname))
+            try: 
+                os.remove(dbname)
             except:
                 pass
 

src/sessions/twitter/compose.py

@@ -1,11 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import absolute_import
-from __future__ import unicode_literals
-from future import standard_library
-standard_library.install_aliases()
-from builtins import str
-from builtins import chr
-from builtins import range
 import platform
 system = platform.system()
 from . import utils
@@ -20,7 +13,6 @@ import config
 from .long_tweets import twishort, tweets
 log = logging.getLogger("compose")
 
-
 def StripChars(s):
     """Converts any html entities in s to their unicode-decoded equivalents and returns a string."""
     entity_re = re.compile(r"&(#\d+|\w+);")
@@ -39,129 +31,124 @@ chars = "abcdefghijklmnopqrstuvwxyz"
 def compose_tweet(tweet, db, relative_times, show_screen_names=False, session=None):
     """ It receives a tweet and returns a list with the user, text for the tweet or message, date and the client where user is."""
     if system == "Windows":
-  original_date = arrow.get(tweet["created_at"], "ddd MMM DD H:m:s Z YYYY", locale="en")
+        original_date = arrow.get(tweet.created_at, locale="en")
         if relative_times == True:
             ts = original_date.humanize(locale=languageHandler.curLang[:2])
         else:
             ts = original_date.shift(seconds=db["utc_offset"]).format(_(u"dddd, MMMM D, YYYY H:m:s"), locale=languageHandler.curLang[:2])
     else:
-  ts = tweet["created_at"]
- if "message" in tweet:
+        ts = tweet.created_at
+    if hasattr(tweet, "message"):
         value = "message"
- elif "full_text" in tweet:
+    elif hasattr(tweet, "full_text"):
         value = "full_text"
     else:
         value = "text"
- if "retweeted_status" in tweet and value != "message":
-  text = StripChars(tweet["retweeted_status"][value])
+    if hasattr(tweet, "retweeted_status") and value != "message":
+        text = StripChars(getattr(tweet.retweeted_status, value))
     else:
-  text = StripChars(tweet[value])
+        text = StripChars(getattr(tweet, value))
     if show_screen_names:
-  user = tweet["user"]["screen_name"]
+        user = session.get_user(tweet.user).screen_name
     else:
-  user = tweet["user"]["name"]
- source = re.sub(r"(?s)<.*?>", "", tweet["source"])
- if "retweeted_status" in tweet:
-  if ("message" in tweet) == False and tweet["retweeted_status"]["is_quote_status"] == False:
-   text = "RT @%s: %s" % (tweet["retweeted_status"]["user"]["screen_name"], text)
-  elif tweet["retweeted_status"]["is_quote_status"]:
+        user = session.get_user(tweet.user).name
+    source = re.sub(r"(?s)<.*?>", "", tweet.source)
+    if hasattr(tweet, "retweeted_status"):
+        if hasattr(tweet, "message") == False and hasattr(tweet.retweeted_status, "is_quote_status") == False:
+            text = "RT @%s: %s" % (session.get_user(tweet.retweeted_status.user).screen_name, text)
+        elif hasattr(tweet.retweeted_status, "is_quote_status"):
             text = "%s" % (text)
         else:
-   text = "RT @%s: %s" % (tweet["retweeted_status"]["user"]["screen_name"], text)
- if ("message" in tweet) == False:
-  urls = utils.find_urls_in_text(text)
-  if "retweeted_status" in tweet:
-   for url in range(0, len(urls)):
-    try:
-     text = text.replace(urls[url], tweet["retweeted_status"]["entities"]["urls"][url]["expanded_url"])
-    except: pass
+            text = "RT @%s: %s" % (session.get_user(tweet.retweeted_status.user).screen_name, text)
+    if not hasattr(tweet, "message"):
+        if hasattr(tweet, "retweeted_status"):
+            if hasattr(tweet.retweeted_status, "entities"):
+                text = utils.expand_urls(text, tweet.retweeted_status.entities)
         else:
-   for url in range(0, len(urls)):
-    try:
-     text = text.replace(urls[url], tweet["entities"]["urls"][url]["expanded_url"])
-    except: pass
+            if hasattr(tweet, "entities"):
+                text = utils.expand_urls(text, tweet.entities)
         if config.app['app-settings']['handle_longtweets']: pass
     return [user+", ", text, ts+", ", source]
 
 def compose_direct_message(item, db, relative_times, show_screen_names=False, session=None):
- # for a while this function will be together with compose_dm.
- # this one composes direct messages based on events (new API Endpoints).
     if system == "Windows":
         # Let's remove the last 3 digits in the timestamp string.
         # Twitter sends their "epoch" timestamp with 3 digits for milliseconds and arrow doesn't like it.
-  original_date = arrow.get(int(item["created_timestamp"][:-3]))
+        original_date = arrow.get(int(item.created_timestamp))
         if relative_times == True:
             ts = original_date.humanize(locale=languageHandler.curLang[:2])
         else:
             ts = original_date.shift(seconds=db["utc_offset"]).format(_(u"dddd, MMMM D, YYYY H:m:s"), locale=languageHandler.curLang[:2])
     else:
-  ts = item["created_timestamp"]
- text = StripChars(item["message_create"]["message_data"]["text"])
+        ts = item.created_timestamp
+    text = StripChars(item.message_create["message_data"]["text"])
     source = "DM"
- sender = session.get_user(item["message_create"]["sender_id"])
- if db["user_name"] == sender["screen_name"]:
+    sender = session.get_user(item.message_create["sender_id"])
+    if db["user_name"] == sender.screen_name:
         if show_screen_names:
-   user = _(u"Dm to %s ") % (session.get_user(item["message_create"]["target"]["recipient_id"])["screen_name"])
+            user = _(u"Dm to %s ") % (session.get_user(item.message_create["target"]["recipient_id"]).screen_name)
         else:
-   user = _(u"Dm to %s ") % (session.get_user(item["message_create"]["target"]["recipient_id"])["name"])
+            user = _(u"Dm to %s ") % (session.get_user(item.message_create["target"]["recipient_id"]).name)
     else:
         if show_screen_names:
-   user = sender["screen_name"]
+            user = sender.screen_name
         else:
-   user = sender["name"]
+            user = sender.name
     if text[-1] in chars: text=text+"."
- urls = utils.find_urls_in_text(text)
- for url in range(0, len(urls)):
-  try:  text = text.replace(urls[url], item["message_create"]["message_data"]["entities"]["urls"][url]["expanded_url"])
-  except IndexError: pass
+    text = utils.expand_urls(text, item.message_create["message_data"]["entities"])
     return [user+", ", text, ts+", ", source]
 
 def compose_quoted_tweet(quoted_tweet, original_tweet, show_screen_names=False, session=None):
     """ It receives a tweet and returns a list with the user, text for the tweet or message, date and the client where user is."""
- if "retweeted_status" in quoted_tweet:
-  if "full_text" in quoted_tweet["retweeted_status"]:
+    if hasattr(quoted_tweet, "retweeted_status"):
+        if hasattr(quoted_tweet.retweeted_status, "full_text"):
             value = "full_text"
         else:
             value = "text"
-  text = StripChars(quoted_tweet["retweeted_status"][value])
+        text = StripChars(getattr(quoted_tweet.retweeted_status, value))
     else:
-  if "full_text" in quoted_tweet:
+        if hasattr(quoted_tweet, "full_text"):
             value = "full_text"
         else:
             value = "text"
-  text = StripChars(quoted_tweet[value])
+        text = StripChars(getattr(quoted_tweet, value))
     if show_screen_names:
-  quoting_user = quoted_tweet["user"]["screen_name"]
+        quoting_user = session.get_user(quoted_tweet.user).screen_name
     else:
-  quoting_user = quoted_tweet["user"]["name"]
- source = re.sub(r"(?s)<.*?>", "", quoted_tweet["source"])
- if "retweeted_status" in quoted_tweet:
-  text = "rt @%s: %s" % (quoted_tweet["retweeted_status"]["user"]["screen_name"], text)
+        quoting_user = session.get_user(quoted_tweet.user).name
+    source = quoted_tweet.source
+    if hasattr(quoted_tweet, "retweeted_status"):
+        text = "rt @%s: %s" % (session.get_user(quoted_tweet.retweeted_status.user).screen_name, text)
     if text[-1] in chars: text=text+"."
- original_user = original_tweet["user"]["screen_name"]
- if "message" in original_tweet:
-  original_text = original_tweet["message"]
- elif "full_text" in original_tweet:
-  original_text = StripChars(original_tweet["full_text"])
+    original_user = session.get_user(original_tweet.user).screen_name
+    if hasattr(original_tweet, "message"):
+        original_text = original_tweet.message
+    elif hasattr(original_tweet, "full_text"):
+        original_text = StripChars(original_tweet.full_text)
     else:
-   original_text = StripChars(original_tweet["text"])
- quoted_tweet["message"] = _(u"{0}. Quoted  tweet from @{1}: {2}").format( text, original_user, original_text)
+        original_text = StripChars(original_tweet.text)
+    quoted_tweet.message = _(u"{0}. Quoted  tweet from @{1}: {2}").format( text, original_user, original_text)
     quoted_tweet = tweets.clear_url(quoted_tweet)
- quoted_tweet["entities"]["urls"].extend(original_tweet["entities"]["urls"])
+    if hasattr(original_tweet, "entities") and original_tweet.entities.get("urls"):
+        if hasattr(quoted_tweet, "entities") == False:
+            quoted_tweet.entities = {}
+        if quoted_tweet.entities.get("urls") == None:
+            quoted_tweet.entities["urls"] = []
+        quoted_tweet.entities["urls"].extend(original_tweet.entities["urls"])
     return quoted_tweet
 
 def compose_followers_list(tweet, db, relative_times=True, show_screen_names=False, session=None):
     if system == "Windows":
-  original_date = arrow.get(tweet["created_at"], "ddd MMM D H:m:s Z YYYY", locale="en")
+        original_date = arrow.get(tweet.created_at, locale="en")
         if relative_times == True:
             ts = original_date.humanize(locale=languageHandler.curLang[:2])
         else:
             ts = original_date.shift(seconds=db["utc_offset"]).format(_(u"dddd, MMMM D, YYYY H:m:s"), locale=languageHandler.curLang[:2])
     else:
-  ts = tweet["created_at"]
- if "status" in tweet:
-  if len(tweet["status"]) > 4 and system == "Windows":
-   original_date2 = arrow.get(tweet["status"]["created_at"], "ddd MMM D H:m:s Z YYYY", locale="en")
+        ts = tweet.created_at
+    if hasattr(tweet, "status"):
+        if system == "Windows":
+            original_date2 = arrow.get(tweet.status.created_at, locale="en")
             if relative_times:
                 ts2 = original_date2.humanize(locale=languageHandler.curLang[:2])
             else:
@@ -170,14 +157,14 @@ def compose_followers_list(tweet, db, relative_times=True, show_screen_names=Fal
             ts2 = _("Unavailable")
     else:
         ts2 = _("Unavailable")
- return [_(u"%s (@%s). %s followers, %s friends, %s tweets. Last tweeted %s. Joined Twitter %s") % (tweet["name"], tweet["screen_name"], tweet["followers_count"], tweet["friends_count"],  tweet["statuses_count"], ts2, ts)]
+    return [_(u"%s (@%s). %s followers, %s friends, %s tweets. Last tweeted %s. Joined Twitter %s") % (tweet.name, tweet.screen_name, tweet.followers_count, tweet.friends_count,  tweet.statuses_count, ts2, ts)]
 
 def compose_list(list):
- name = list["name"]
- if list["description"] == None: description = _(u"No description available")
- else: description = list["description"]
- user = list["user"]["name"]
- members = str(list["member_count"])
- if list["mode"] == "private": status = _(u"private")
+    name = list.name
+    if list.description == None: description = _(u"No description available")
+    else: description = list.description
+    user = list.user.name
+    members = str(list.member_count)
+    if list.mode == "private": status = _(u"private")
     else: status = _(u"public")
     return [name, description, user, members, status]

src/sessions/twitter/long_tweets/tweets.py

@@ -23,30 +23,30 @@ def is_long(tweet):
     """ Check if the passed tweet contains a quote in its metadata.
      tweet dict: a tweet dictionary.
      returns True if a quote is detected, False otherwise."""
- if "quoted_status_id" in tweet and "quoted_status" in tweet:
-  return tweet["quoted_status_id"]
- elif "retweeted_status" in tweet and "quoted_status_id" in tweet["retweeted_status"] and "quoted_status" in tweet["retweeted_status"]:
-  return tweet["retweeted_status"]["quoted_status_id"]
+    if hasattr(tweet, "quoted_status_id") and hasattr(tweet, "quoted_status"):
+        return tweet.quoted_status_id
+    elif hasattr(tweet, "retweeted_status")  and hasattr(tweet.retweeted_status, "quoted_status_id") and hasattr(tweet.retweeted_status, "quoted_status"):
+        return tweet.retweeted_status.quoted_status_id
     return False
 
 def clear_url(tweet):
     """ Reads data from a quoted tweet and removes the link to the Status from the tweet's text.
      tweet dict: a tweet dictionary.
      returns a tweet dictionary without the URL to the status ID in its text to display."""
- if "retweeted_status" in tweet:
-  if "full_text" in tweet["retweeted_status"]:
+    if hasattr(tweet, "retweeted_status"):
+        if hasattr(tweet.retweeted_status, "full_text"):
             value = "full_text"
         else:
             value = "text"
-  urls = utils.find_urls_in_text(tweet["retweeted_status"][value])
-  try: tweet["message"] = tweet["message"].replace(urls[-1], "")
+        urls = utils.find_urls_in_text(getattr(tweet.retweeted_status, value))
+        try: tweet.message = tweet.message.replace(urls[-1], "")
         except IndexError: pass
     else:
-  if "full_text" in tweet:
+        if hasattr(tweet, "full_text"):
             value = "full_text"
         else:
             value = "text"
-  urls = utils.find_urls_in_text(tweet[value])
-  try: tweet["message"] = tweet["message"].replace(urls[-1], "")
+        urls = utils.find_urls_in_text(getattr(tweet, value))
+        try: tweet.message = tweet.message.replace(urls[-1], "")
         except IndexError: pass
     return tweet

src/sessions/twitter/long_tweets/twishort.py

@@ -40,21 +40,21 @@ def is_long(tweet):
     """ Check if the passed tweet is made with Twishort.
      returns True if is a long tweet, False otherwise."""
     long = False
- for url in range(0, len(tweet["entities"]["urls"])):
+    for url in range(0, len(tweet.entities["urls"])):
         try:
-   if tweet["entities"]["urls"][url] != None and "twishort.com" in tweet["entities"]["urls"][url]["expanded_url"]:
-    long = get_twishort_uri(tweet["entities"]["urls"][url]["expanded_url"])
+            if tweet.entities["urls"][url] != None and "twishort.com" in tweet.entities["urls"][url]["expanded_url"]:
+                long = get_twishort_uri(tweet.entities["urls"][url]["expanded_url"])
         except IndexError:
             pass
         # sometimes Twitter returns URL's with None objects, so let's take it.
         # see https://github.com/manuelcortez/TWBlue/issues/103
         except TypeError:
             pass
- if long == False and "retweeted_status" in tweet:
-  for url in range(0, len(tweet["retweeted_status"]["entities"]["urls"])):
+    if long == False and hasattr(tweet, "retweeted_status"):
+        for url in range(0, len(tweet.retweeted_status.entities["urls"])):
             try:
-    if tweet["retweeted_status"]["entities"]["urls"][url] != None and "twishort.com" in tweet["retweeted_status"]["entities"]["urls"][url]["expanded_url"]:
-     long = get_twishort_uri(tweet["retweeted_status"]["entities"]["urls"][url]["expanded_url"])
+                if tweet.retweeted_status.entities["urls"][url] != None and "twishort.com" in tweet.retweeted_status.entities["urls"][url]["expanded_url"]:
+                    long = get_twishort_uri(tweet.retweeted_status.entities["urls"][url]["expanded_url"])
             except IndexError:
                 pass
             except TypeError:

src/sessions/twitter/reduce.py

@@ -0,0 +1,37 @@
+# -*- coding: utf-8 -*-
+""" Strips unneeded tweet information in order to store tweet objects by using less memory. This is especially useful when buffers start to contain more than a certain amount of items. """
+from tweepy.models import Status
+
+def reduce_tweet(tweet):
+    """ generates a new Tweet model with the fields we currently need, excluding everything else  including null values and empty collections. """
+    allowed_values = ["created_at", "id", "full_text", "text", "message", "in_reply_to_status_id", "in_reply_to_user_id", "is_quote_status", "lang", "source", "coordinates", "quoted_status_id", "extended_entities"]
+    allowed_entities = ["hashtags", "media", "urls", "user_mentions", "polls"]
+    status_dict = {}
+    for key in allowed_values:
+        if tweet._json.get(key):
+            status_dict[key] = tweet._json[key]
+    entities = dict()
+    for key in allowed_entities:
+        if tweet._json["entities"].get(key) and tweet._json["entities"].get(key) != None:
+            entities[key] = tweet._json["entities"][key]
+    status_dict["entities"] = entities
+    # If tweet comes from the cached database, it does not include an API,  so we can pass None here as we do not use that reference to tweepy's API.
+    if hasattr(tweet, "_api"):
+        api = tweet._api
+    else:
+        api = None
+    status = Status().parse(api=api, json=status_dict)
+    # Quotes and retweets are different objects. So we parse a new tweet when we have a quoted or retweeted status here.
+    if tweet._json.get("quoted_status"):
+        quoted_tweet = reduce_tweet(tweet.quoted_status)
+        status.quoted_status = quoted_tweet
+    if tweet._json.get("retweeted_status"):
+        retweeted_tweet = reduce_tweet(tweet.retweeted_status)
+        status.retweeted_status = retweeted_tweet
+    # Adds user ID to here so we can reference it later.
+    # Sometimes, the conversations buffer would send an already reduced tweet here so we will need to return it as is.
+    if isinstance(tweet.user, str) == False:
+        status.user = tweet.user.id_str
+    else:
+        return tweet
+    return status

src/sessions/twitter/session.py

@@ -1,8 +1,5 @@
 # -*- coding: utf-8 -*-
 """ This is the main session needed to access all Twitter Features."""
-from __future__ import absolute_import
-from __future__ import unicode_literals
-from builtins import range
 import os
 import time
 import logging
@@ -12,12 +9,15 @@ import config
 import output
 import application
 from pubsub import pub
-from twython import Twython, TwythonError, TwythonRateLimitError, TwythonAuthError
+import tweepy
+from tweepy.error import TweepError
+from tweepy.models import User as UserModel
 from mysc.thread_utils import call_threaded
 from keys import keyring
 from sessions import base
 from sessions.twitter import utils, compose
 from sessions.twitter.long_tweets import tweets, twishort
+from . import reduce
 from .wxUI import authorisationDialog
 
 log = logging.getLogger("sessions.twitterSession")
@@ -31,52 +31,52 @@ class Session(base.baseSession):
         data list: A list with tweets.
         ignore_older bool: if set to True, items older than the first element on the list will be ignored.
         returns the number of items that have been added in this execution"""
+        if name == "direct_messages":
+            return self.order_direct_messages(data)
         num = 0
         last_id = None
         if (name in self.db) == False:
             self.db[name] = []
         if ("users" in self.db) == False:
             self.db["users"] = {}
+        objects = self.db[name]
         if ignore_older and len(self.db[name]) > 0:
             if self.settings["general"]["reverse_timelines"] == False:
-    last_id = self.db[name][0]["id"]
+                last_id = self.db[name][0].id
             else:
-    last_id = self.db[name][-1]["id"]
+                last_id = self.db[name][-1].id
+        self.add_users_from_results(data)
         for i in data:
             if ignore_older and last_id != None:
-    if i["id"] < last_id:
-     log.error("Ignoring an older tweet... Last id: {0}, tweet id: {1}".format(last_id, i["id"]))
+                if i.id < last_id:
+                    log.error("Ignoring an older tweet... Last id: {0}, tweet id: {1}".format(last_id, i.id))
                     continue
-   if utils.find_item(i["id"], self.db[name]) == None and     utils.is_allowed(i, self.settings, name) == True:
-    i = self.check_quoted_status(i)
-    i = self.check_long_tweet(i)
+            if utils.find_item(i.id, self.db[name]) == None and     utils.is_allowed(i, self.settings, name) == True:
                 if i == False: continue
-    if self.settings["general"]["reverse_timelines"] == False: self.db[name].append(i)
-    else: self.db[name].insert(0, i)
+                reduced_object = reduce.reduce_tweet(i)
+                reduced_object = self.check_quoted_status(reduced_object)
+                reduced_object = self.check_long_tweet(reduced_object)
+                if self.settings["general"]["reverse_timelines"] == False: objects.append(reduced_object)
+                else: objects.insert(0, reduced_object)
                 num = num+1
-    if ("user" in i) == True:
-     if (i["user"]["id"] in self.db["users"]) == False:
-      self.db["users"][i["user"]["id"]] = i["user"]
+        self.db[name] = objects
         return num
 
- def order_cursored_buffer(self, name, data):
+    def order_people(self, name, data):
         """ Put new items on the local database. Useful for cursored buffers (followers, friends, users of a list and searches)
         name str: The name for the buffer stored in the dictionary.
         data list: A list with items and some information about cursors.
         returns the number of items that have been added in this execution"""
-  # Direct messages should be added to db in other function.
-  # Because they will be populating two buffers with one endpoint.
-  if name == "direct_messages":
-   return self.order_direct_messages(data)
         num = 0
         if (name in self.db) == False:
-   self.db[name] = {}
-   self.db[name]["items"] = []
+            self.db[name] = []
+        objects = self.db[name]
         for i in data:
-   if utils.find_item(i["id"], self.db[name]["items"]) == None:
-    if self.settings["general"]["reverse_timelines"] == False: self.db[name]["items"].append(i)
-    else: self.db[name]["items"].insert(0, i)
+            if utils.find_item(i.id, self.db[name]) == None:
+                if self.settings["general"]["reverse_timelines"] == False: objects.append(i)
+                else: objects.insert(0, i)
                 num = num+1
+        self.db[name] = objects
         return num
 
     def order_direct_messages(self, data):
@@ -86,27 +86,45 @@ class Session(base.baseSession):
         incoming = 0
         sent = 0
         if ("direct_messages" in self.db) == False:
-   self.db["direct_messages"] = {}
-   self.db["direct_messages"]["items"] = []
+            self.db["direct_messages"] = []
+        if ("sent_direct_messages" in self.db) == False:
+            self.db["sent_direct_messages"] = []
+        objects = self.db["direct_messages"]
+        sent_objects = self.db["sent_direct_messages"]
         for i in data:
-   if i["message_create"]["sender_id"] == self.db["user_id"]:
-    if "sent_direct_messages" in self.db and utils.find_item(i["id"], self.db["sent_direct_messages"]["items"]) == None:
-     if self.settings["general"]["reverse_timelines"] == False: self.db["sent_direct_messages"]["items"].append(i)
-     else: self.db["sent_direct_messages"]["items"].insert(0, i)
+            # Twitter returns sender_id as str, which must be converted to int in order to match to our user_id object.
+            if int(i.message_create["sender_id"]) == self.db["user_id"]:
+                if "sent_direct_messages" in self.db and utils.find_item(i.id, self.db["sent_direct_messages"]) == None:
+                    if self.settings["general"]["reverse_timelines"] == False: sent_objects.append(i)
+                    else: sent_objects.insert(0, i)
                     sent = sent+1
             else:
-    if utils.find_item(i["id"], self.db["direct_messages"]["items"]) == None:
-     if self.settings["general"]["reverse_timelines"] == False: self.db["direct_messages"]["items"].append(i)
-     else: self.db["direct_messages"]["items"].insert(0, i)
+                if utils.find_item(i.id, self.db["direct_messages"]) == None:
+                    if self.settings["general"]["reverse_timelines"] == False: objects.append(i)
+                    else: objects.insert(0, i)
                     incoming = incoming+1
+        self.db["direct_messages"] = objects
+
+        self.db["sent_direct_messages"] = sent_objects
         pub.sendMessage("sent-dms-updated", total=sent, account=self.db["user_name"])
+
+
         return incoming
 
     def __init__(self, *args, **kwargs):
         super(Session, self).__init__(*args, **kwargs)
+        # Adds here the optional cursors objects.
+        cursors = dict(direct_messages=-1)
+        self.db["cursors"] = cursors
         self.reconnection_function_active = False
         self.counter = 0
         self.lists = []
+        # As users are cached for accessing them with not too many twitter calls,
+        # there could be a weird situation where a deleted user who sent direct messages to the current account will not be able to be retrieved at twitter.
+        # So we need to store an "user deleted" object in the cache, but have the ID of the deleted user in a local reference.
+        # This will be especially useful because if the user reactivates their account later, TWblue will try to retrieve such user again at startup.
+        # If we wouldn't implement this approach, TWBlue would save permanently the "deleted user" object.
+        self.deleted_users = {}
 
 # @_require_configuration
     def login(self, verify_credentials=True):
@@ -115,7 +133,9 @@ class Session(base.baseSession):
         if self.settings["twitter"]["user_key"] != None and self.settings["twitter"]["user_secret"] != None:
             try:
                 log.debug("Logging in to twitter...")
-    self.twitter = Twython(keyring.get("api_key"), keyring.get("api_secret"), self.settings["twitter"]["user_key"], self.settings["twitter"]["user_secret"])
+                self.auth = tweepy.OAuthHandler(keyring.get("api_key"), keyring.get("api_secret"))
+                self.auth.set_access_token(self.settings["twitter"]["user_key"], self.settings["twitter"]["user_secret"])
+                self.twitter = tweepy.API(self.auth)
                 if verify_credentials == True:
                     self.credentials = self.twitter.verify_credentials()
                 self.logged = True
@@ -134,19 +154,18 @@ class Session(base.baseSession):
         if self.logged == True:
             raise Exceptions.AlreadyAuthorisedError("The authorisation process is not needed at this time.")
         else:
-   twitter = Twython(keyring.get("api_key"), keyring.get("api_secret"))
-   self.auth = twitter.get_authentication_tokens(callback_url="oob")
-   webbrowser.open_new_tab(self.auth['auth_url'])
+            self.auth = tweepy.OAuthHandler(keyring.get("api_key"), keyring.get("api_secret"))
+            redirect_url = self.auth.get_authorization_url()
+            webbrowser.open_new_tab(redirect_url)
             self.authorisation_dialog = authorisationDialog()
             self.authorisation_dialog.cancel.Bind(wx.EVT_BUTTON, self.authorisation_cancelled)
             self.authorisation_dialog.ok.Bind(wx.EVT_BUTTON, self.authorisation_accepted)
             self.authorisation_dialog.ShowModal()
 
     def verify_authorisation(self, pincode):
-  twitter = Twython(keyring.get("api_key"), keyring.get("api_secret"), self.auth['oauth_token'], self.auth['oauth_token_secret'])
-  final = twitter.get_authorized_tokens(pincode)
-  self.settings["twitter"]["user_key"] = final["oauth_token"]
-  self.settings["twitter"]["user_secret"] = final["oauth_token_secret"]
+        self.auth.get_access_token(pincode)
+        self.settings["twitter"]["user_key"] = self.auth.access_token
+        self.settings["twitter"]["user_secret"] = self.auth.access_token_secret
         self.settings.write()
         del self.auth
 
@@ -161,35 +180,6 @@ class Session(base.baseSession):
         self.verify_authorisation(pincode)
         self.authorisation_dialog.Destroy()
 
- def get_more_items(self, update_function, users=False, dm=False, name=None, *args, **kwargs):
-  """ Get more items for twitter objects.
-  update_function str: function to call for getting more items. Must be member of self.twitter.
-  users, dm bool: If any of these is set to True, the function will treat items as users or dm (they need different handling).
-  name str: name of the database item to put new element in."""
-  results = []
-  if "cursor" in kwargs and kwargs["cursor"] == 0:
-   output.speak(_(u"There are no more items to retrieve in this buffer."))
-   return
-  data = getattr(self.twitter, update_function)(*args, **kwargs)
-  if users == True:
-   if type(data) == dict and "next_cursor" in data:
-    if "next_cursor" in data: # There are more objects to retrieve.
-     self.db[name]["cursor"] = data["next_cursor"]
-    else: # Set cursor to 0, wich means no more items available.
-     self.db[name]["cursor"] = 0
-    for i in data["users"]: results.append(i)
-   elif type(data) == list:
-    results.extend(data[1:])
-  elif dm == True:
-   if "next_cursor" in data: # There are more objects to retrieve.
-    self.db[name]["cursor"] = data["next_cursor"]
-   else: # Set cursor to 0, wich means no more items available.
-    self.db[name]["cursor"] = 0
-   for i in data["events"]: results.append(i)
-  else:
-   results.extend(data[1:])
-  return results
-
     def api_call(self, call_name, action="", _sound=None, report_success=False, report_failure=True, preexec_message="", *args, **kwargs):
         """ Make a call to the Twitter API. If there is a connectionError or another exception not related to Twitter, It will call the method again at least 25 times, waiting a while between calls. Useful for  post methods.
         If twitter returns an error, it will not call the method anymore.
@@ -207,14 +197,14 @@ class Session(base.baseSession):
             try:
                 val = getattr(self.twitter, call_name)(*args, **kwargs)
                 finished = True
-   except TwythonError as e:
-    output.speak(e.msg)
+            except TweepError as e:
+                output.speak(e.reason)
                 val = None
                 if e.error_code != 403 and e.error_code != 404:
                     tries = tries+1
                     time.sleep(5)
-    elif report_failure and hasattr(e, 'message'):
-     output.speak(_("%s failed.  Reason: %s") % (action, e.msg))
+                elif report_failure and hasattr(e, 'reason'):
+                    output.speak(_("%s failed.  Reason: %s") % (action, e.reason))
                 finished = True
 #   except:
 #    tries = tries + 1
@@ -227,15 +217,15 @@ class Session(base.baseSession):
     def search(self, name, *args, **kwargs):
         """ Search in twitter, passing args and kwargs as arguments to the Twython function."""
         tl = self.twitter.search(*args, **kwargs)
-  tl["statuses"].reverse()
-  return tl["statuses"]
+        tl.reverse()
+        return tl
 
 # @_require_login
     def get_favourites_timeline(self, name, *args, **kwargs):
         """ Gets favourites for the authenticated user or a friend or follower.
         name str: Name for storage in the database.
         args and kwargs are passed directly to the Twython function."""
-  tl = self.call_paged("get_favorites", *args, **kwargs)
+        tl = self.call_paged("favorites", *args, **kwargs)
         return self.order_buffer(name, tl)
 
     def call_paged(self, update_function, *args, **kwargs):
@@ -249,8 +239,8 @@ class Session(base.baseSession):
         data = getattr(self.twitter, update_function)(count=self.settings["general"]["max_tweets_per_call"], *args, **kwargs)
         results.extend(data)
         for i in range(0, max):
-   if i == 0: max_id = results[-1]["id"]
-   else: max_id = results[0]["id"]
+            if i == 0: max_id = results[-1].id
+            else: max_id = results[0].id
             data = getattr(self.twitter, update_function)(max_id=max_id, count=self.settings["general"]["max_tweets_per_call"], *args, **kwargs)
             results.extend(data)
         results.reverse()
@@ -259,11 +249,11 @@ class Session(base.baseSession):
 # @_require_login
     def get_user_info(self):
         """ Retrieves some information required by TWBlue for setup."""
-  f = self.twitter.get_account_settings()
+        f = self.twitter.get_settings()
         sn = f["screen_name"]
         self.settings["twitter"]["user_name"] = sn
         self.db["user_name"] = sn
-  self.db["user_id"] = self.twitter.show_user(screen_name=sn)["id_str"]
+        self.db["user_id"] = self.twitter.get_user(screen_name=sn).id
         try:
             self.db["utc_offset"] = f["time_zone"]["utc_offset"]
         except KeyError:
@@ -271,7 +261,7 @@ class Session(base.baseSession):
         # Get twitter's supported languages and save them in a global variable
         #so we won't call to this method once per session.
         if len(application.supported_languages) == 0:
-   application.supported_languages = self.twitter.get_supported_languages()
+            application.supported_languages = self.twitter.supported_languages()
         self.get_lists()
         self.get_muted_users()
         self.settings.write()
@@ -279,12 +269,12 @@ class Session(base.baseSession):
 # @_require_login
     def get_lists(self):
         """ Gets the lists that the user is subscribed to and stores them in the database. Returns None."""
-  self.db["lists"] = self.twitter.show_lists(reverse=True)
+        self.db["lists"] = self.twitter.lists_all(reverse=True)
 
 # @_require_login
     def get_muted_users(self):
         """ Gets muted users (oh really?)."""
-  self.db["muted_users"] = self.twitter.list_mute_ids()["ids"]
+        self.db["muted_users"] = self.twitter.mutes_ids()
 
 # @_require_login
     def get_stream(self, name, function, *args, **kwargs):
@@ -355,74 +345,93 @@ class Session(base.baseSession):
     def get_quoted_tweet(self, tweet):
         """ Process a tweet and extract all information related to the quote. """
         quoted_tweet = tweet
-  if "full_text" in tweet:
+        if hasattr(tweet, "full_text"):
             value = "full_text"
         else:
             value = "text"
-  urls = utils.find_urls_in_text(quoted_tweet[value])
-  for url in range(0, len(urls)):
-   try:  quoted_tweet[value] = quoted_tweet[value].replace(urls[url], quoted_tweet["entities"]["urls"][url]["expanded_url"])
-   except IndexError: pass
-  if "quoted_status" in quoted_tweet:
-   original_tweet = quoted_tweet["quoted_status"]
-  elif "retweeted_status" in quoted_tweet and "quoted_status" in quoted_tweet["retweeted_status"]:
-   original_tweet = quoted_tweet["retweeted_status"]["quoted_status"]
+        if hasattr(quoted_tweet, "entities"):
+            setattr(quoted_tweet, value, utils.expand_urls(getattr(quoted_tweet, value), quoted_tweet.entities))
+        if hasattr(quoted_tweet, "is_quote_status") == True and hasattr(quoted_tweet, "quoted_status"):
+            original_tweet = quoted_tweet.quoted_status
+        elif hasattr(quoted_tweet, "retweeted_status") and hasattr(quoted_tweet.retweeted_status, "is_quote_status") == True and hasattr(quoted_tweet.retweeted_status, "quoted_status"):
+            original_tweet = quoted_tweet.retweeted_status.quoted_status
         else:
             return quoted_tweet
         original_tweet = self.check_long_tweet(original_tweet)
-
-  if "full_text" in original_tweet:
+        if hasattr(original_tweet, "full_text"):
             value = "full_text"
-  elif "message" in original_tweet:
+        elif hasattr(original_tweet, "message"):
             value = "message"
         else:
             value = "text"
-  urls = utils.find_urls_in_text(original_tweet[value])
-  for url in range(0, len(urls)):
-   try:  original_tweet[value] = original_tweet[value].replace(urls[url], original_tweet["entities"]["urls"][url]["expanded_url"])
-   except IndexError: pass
-  return compose.compose_quoted_tweet(quoted_tweet, original_tweet)
+        if hasattr(original_tweet, "entities"):
+            setattr(original_tweet, value, utils.expand_urls(getattr(original_tweet, value), original_tweet.entities))
+        # ToDo: Shall we check whether we should add show_screen_names here?
+        return compose.compose_quoted_tweet(quoted_tweet, original_tweet, session=self)
 
     def check_long_tweet(self, tweet):
         """ Process a tweet and add extra info if it's a long tweet made with Twyshort.
         tweet dict: a tweet object.
         returns a tweet with a new argument message, or original tweet if it's not a long tweet."""
+        long = False
+        if hasattr(tweet, "entities") and tweet.entities.get("urls"):
             long = twishort.is_long(tweet)
         if long != False and config.app["app-settings"]["handle_longtweets"]:
             message = twishort.get_full_text(long)
-   if "quoted_status" in tweet:
-    tweet["quoted_status"]["message"] = message
-    if tweet["quoted_status"]["message"] == False: return False
-    tweet["quoted_status"]["twishort"] = True
-    for i in tweet["quoted_status"]["entities"]["user_mentions"]:
-     if "@%s" % (i["screen_name"]) not in tweet["quoted_status"]["message"] and i["screen_name"] != tweet["user"]["screen_name"]:
-      if "retweeted_status" in tweet["quoted_status"] and tweet["retweeted_status"]["user"]["screen_name"] == i["screen_name"]:
+            if hasattr(tweet, "quoted_status"):
+                tweet.quoted_status.message = message
+                if tweet.quoted_status.message == False: return False
+                tweet.quoted_status.twishort = True
+                if hasattr(tweet.quoted_status, "entities") and tweet.quoted_status.entities.get("user_mentions"):
+                    for i in tweet.quoted_status.entities["user_mentions"]:
+                        if "@%s" % (i["screen_name"]) not in tweet.quoted_status.message and i["screen_name"] != self.get_user(tweet.user).screen_name:
+                            if hasattr(tweet.quoted_status, "retweeted_status")  and self.get_user(tweet.retweeted_status.user).screen_name == i["screen_name"]:
                                 continue
-     tweet["quoted_status"]["message"] = u"@%s %s" % (i["screen_name"], tweet["message"])
+                        tweet.quoted_status.message = u"@%s %s" % (i["screen_name"], tweet.message)
             else:
-    tweet["message"] = message
-    if tweet["message"] == False: return False
-    tweet["twishort"] = True
-    for i in tweet["entities"]["user_mentions"]:
-     if "@%s" % (i["screen_name"]) not in tweet["message"] and i["screen_name"] != tweet["user"]["screen_name"]:
-      if "retweeted_status" in tweet and tweet["retweeted_status"]["user"]["screen_name"] == i["screen_name"]:
+                tweet.message = message
+                if tweet.message == False: return False
+                tweet.twishort = True
+                if hasattr(tweet, "entities") and tweet.entities.get("user_mentions"):
+                    for i in tweet.entities["user_mentions"]:
+                        if "@%s" % (i["screen_name"]) not in tweet.message and i["screen_name"] != self.get_user(tweet.user).screen_name:
+                            if hasattr(tweet, "retweeted_status") and self.get_user(tweet.retweeted_status.user).screen_name == i["screen_name"]:
                                 continue
+                            tweet.message = u"@%s %s" % (i["screen_name"], tweet.message)
         return tweet
 
     def get_user(self, id):
         """ Returns an user object associated with an ID.
         id str: User identifier, provided by Twitter.
-  returns an user dict."""
-  if ("users" in self.db) == False or (id in self.db["users"]) == False:
+        returns a tweepy user object."""
+        if hasattr(id, "id_str"):
+            log.error("Called get_user function by passing a full user id as a parameter.")
+            id = id.id_str
+        # Check if the user has been added to the list of deleted users previously.
+        if id in self.deleted_users:
+            log.debug("Returning user {} from the list of deleted users.".format(id))
+            return self.deleted_users[id]
+        if ("users" in self.db) == False or (str(id) in self.db["users"]) == False:
+            log.debug("Requesting user id {} as it is not present in the users database.".format(id))
             try:
-    user = self.twitter.show_user(id=id)
-   except TwythonError:
-    user = dict(screen_name="deleted_account", name="Deleted account")
-    return user
-   self.db["users"][user["id_str"]] = user
+                user = self.twitter.get_user(id=id)
+            except TweepError as err:
+                user = UserModel(None)
+                user.screen_name = "deleted_user"
+                user.id = id
+                user.name = _("Deleted account")
+                if hasattr(err, "api_code") and err.api_code == 50:
+                    self.deleted_users[id] = user
                     return user
                 else:
-   return self.db["users"][id]
+                    log.exception("Error when attempting to retrieve an user from Twitter.")
+                    return user
+            users = self.db["users"]
+            users[user.id_str] = user
+            self.db["users"] = users
+            return user
+        else:
+            return self.db["users"][str(id)]
 
     def get_user_by_screen_name(self, screen_name):
         """ Returns an user identifier associated with a screen_name.
@@ -430,12 +439,65 @@ class Session(base.baseSession):
         returns an user ID."""
         if ("users" in self.db) == False:
             user = utils.if_user_exists(self.twitter, screen_name)
-   self.db["users"][user["id_str"]] = user
-   return user["id_str"]
+            users = self.db["users"]
+            users[user["id"]] = user
+            self.db["users"] = users
+            return user["id"]
         else:
             for i in list(self.db["users"].keys()):
-    if self.db["users"][i]["screen_name"] == screen_name:
-     return self.db["users"][i]["id_str"]
+                if self.db["users"][i].screen_name == screen_name:
+                    return self.db["users"][i].id
             user = utils.if_user_exists(self.twitter, screen_name)
-   self.db["users"][user["id_str"]] = user
-   return user["id_str"]
+            users = self.db["users"]
+            users[user.id] = user
+            self.db["users"] = users
+            return user.id
+
+    def save_users(self, user_ids):
+        """ Adds all new users to the users database. """
+        if len(user_ids) == 0:
+            return
+        log.debug("Received %d user IDS to be added in the database." % (len(user_ids)))
+        users_to_retrieve = [user_id for user_id in user_ids if (user_id not in self.db["users"] and user_id not in self.deleted_users)]
+        # Remove duplicates
+        users_to_retrieve = list(dict.fromkeys(users_to_retrieve))
+        if len(users_to_retrieve) == 0:
+            return
+        log.debug("TWBlue will get %d new users from Twitter." % (len(users_to_retrieve)))
+        try:
+            users = self.twitter.lookup_users(user_ids=users_to_retrieve, tweet_mode="extended")
+            users_db = self.db["users"]
+            for user in users:
+                users_db[user.id_str] = user
+            log.debug("Added %d new users" % (len(users)))
+            self.db["users"] = users_db
+        except TweepError as err:
+            if hasattr(err, "api_code") and err.api_code == 17: # Users not found.
+                log.error("The specified users {} were not found in twitter.".format(user_ids))
+                # Creates a deleted user object for every user_id not found here.
+                # This will make TWBlue to not waste Twitter API calls when attempting to retrieve those users again.
+                # As deleted_users is not saved across restarts, when restarting TWBlue, it will retrieve the correct users if they enabled their accounts.
+                for id in users_to_retrieve:
+                    user = UserModel(None)
+                    user.screen_name = "deleted_user"
+                    user.id = id
+                    user.name = _("Deleted account")
+                    self.deleted_users[id] = user
+            else:
+                log.exception("An exception happened while attempting to retrieve a list of users from direct messages in Twitter.")
+
+    def add_users_from_results(self, data):
+        users = self.db["users"]
+        for i in data:
+            if hasattr(i, "user"):
+                if isinstance(i.user, str):
+                    log.warning("A String was passed to be added as an user. This is normal only if TWBlue tried to load a conversation.")
+                    continue
+                if (i.user.id_str in self.db["users"]) == False:
+                    users[i.user.id_str] = i.user
+                if hasattr(i, "quoted_status") and (i.quoted_status.user.id_str in self.db["users"]) == False:
+                    users[i.quoted_status.user.id_str] = i.quoted_status.user
+
+                if hasattr(i, "retweeted_status") and (i.retweeted_status.user.id_str in self.db["users"]) == False:
+                    users[i.retweeted_status.user.id_str] = i.retweeted_status.user
+        self.db["users"] = users

src/sessions/twitter/utils.py

@@ -1,16 +1,12 @@
 # -*- coding: utf-8 -*-
-from __future__ import print_function
-from __future__ import unicode_literals
-from builtins import str
-from builtins import range
 import url_shortener, re
 import output
-from twython import TwythonError
 import config
 import logging
 import requests
 import time
 import sound
+from tweepy.error import TweepError
 log = logging.getLogger("twitter.utils")
 """ Some utilities for the twitter interface."""
 
@@ -25,38 +21,39 @@ bad_chars = '\'\\\n.,[](){}:;"'
 def find_urls_in_text(text):
     return  url_re2.findall(text)
 
-def find_urls (tweet):
+def find_urls (tweet, twitter_media=False):
     urls = []
+    if twitter_media and hasattr(tweet, "extended_entities"):
+        for mediaItem in tweet.extended_entities["media"]:
+            if mediaItem["type"] == "video":
+                for variant in mediaItem["video_info"]["variants"]:
+                    if variant["content_type"] == "video/mp4":
+                        urls.append(variant["url"])
+                        break
     # Let's add URLS from tweet entities.
- if "message_create" in tweet:
-  entities = tweet["message_create"]["message_data"]["entities"]
+    if hasattr(tweet, "message_create"):
+        entities = tweet.message_create["message_data"]["entities"]
     else:
-  entities = tweet["entities"]
+        if hasattr(tweet, "entities") == True:
+            entities = tweet.entities
+    if entities.get("urls") != None:
         for i in entities["urls"]:
             if i["expanded_url"] not in urls:
                 urls.append(i["expanded_url"])
- if "quoted_status" in tweet:
-  for i in tweet["quoted_status"]["entities"]["urls"]:
-   if i["expanded_url"] not in urls:
-    urls.append(i["expanded_url"])
- if "retweeted_status" in tweet:
-  for i in tweet["retweeted_status"]["entities"]["urls"]:
-   if i["expanded_url"] not in urls:
-    urls.append(i["expanded_url"])
-  if "quoted_status" in tweet["retweeted_status"]:
-   for i in tweet["retweeted_status"]["quoted_status"]["entities"]["urls"]:
-    if i["expanded_url"] not in urls:
-     urls.append(i["expanded_url"])
- if "message" in tweet:
+    if hasattr(tweet, "quoted_status"):
+        urls.extend(find_urls(tweet.quoted_status))
+    if hasattr(tweet, "retweeted_status"):
+        urls.extend(find_urls(tweet.retweeted_status))
+    if hasattr(tweet, "message"):
         i = "message"
- elif "full_text" in tweet:
+    elif hasattr(tweet, "full_text"):
         i = "full_text"
     else:
         i = "text"
- if "message_create" in tweet:
-  extracted_urls = find_urls_in_text(tweet["message_create"]["message_data"]["text"])
+    if hasattr(tweet, "message_create"):
+        extracted_urls = find_urls_in_text(tweet.message_create["message_data"]["text"])
     else:
-  extracted_urls = find_urls_in_text(tweet[i])
+        extracted_urls = find_urls_in_text(getattr(tweet, i))
     # Don't include t.co links (mostly they are photos or shortened versions of already added URLS).
     for i in extracted_urls:
         if i not in urls and "https://t.co" not in i:
@@ -65,124 +62,112 @@ def find_urls (tweet):
 
 def find_item(id, listItem):
     for i in range(0, len(listItem)):
-  if listItem[i]["id"] == id: return i
+        if listItem[i].id == id: return i
     return None
 
 def find_list(name, lists):
     for i in range(0, len(lists)):
-  if lists[i]["name"] == name:  return lists[i]["id"]
-
-def find_previous_reply(id, listItem):
- for i in range(0, len(listItem)):
-  if listItem[i]["id_str"] == str(id): return i
- return None
-
-def find_next_reply(id, listItem):
- for i in range(0, len(listItem)):
-  if listItem[i]["in_reply_to_status_id_str"] == str(id): return i
- return None
+        if lists[i].name == name:  return lists[i].id
 
 def is_audio(tweet):
+    # Checks firstly for Twitter videos and audios.
+    if hasattr(tweet, "extended_entities"):
+        for mediaItem in tweet.extended_entities["media"]:
+            if mediaItem["type"] == "video":
+                return True
     try:
         if len(find_urls(tweet)) < 1:
             return False
-  if "message_create" in tweet:
-   entities = tweet["message_create"]["message_data"]["entities"]
+        if hasattr(tweet, "message_create"):
+            entities = tweet.message_create["message_data"]["entities"]
         else:
-   entities = tweet["entities"]
+            if hasattr(tweet, "entities") == False or tweet.entities.get("hashtags") == None:
+                return False
+            entities = tweet.entities
         if len(entities["hashtags"]) > 0:
             for i in entities["hashtags"]:
                 if i["text"] == "audio":
                     return True
     except IndexError:
-  print(tweet["entities"]["hashtags"])
         log.exception("Exception while executing is_audio hashtag algorithm")
 
 def is_geocoded(tweet):
- if "coordinates" in tweet and tweet["coordinates"] != None:
+    if hasattr(tweet, "coordinates") and tweet.coordinates != None:
         return True
 
 def is_media(tweet):
- if "message_create" in tweet:
-  entities = tweet["message_create"]["message_data"]["entities"]
+    if hasattr(tweet, "message_create"):
+        entities = tweet.message_create["message_data"]["entities"]
     else:
-  entities = tweet["entities"]
- if ("media" in entities) == False:
+        if hasattr(tweet, "entities") == False or tweet.entities.get("hashtags") == None:
+            return False
+        entities = tweet.entities
+    if entities.get("media") == None:
         return False
     for i in entities["media"]:
-  if "type" in i and i["type"] == "photo":
+        if i.get("type") != None and i.get("type") == "photo":
             return True
     return False
 
 def get_all_mentioned(tweet, conf, field="screen_name"):
- """ Gets all users that has been mentioned."""
+    """ Gets all users that have been mentioned."""
     results = []
- for i in tweet["entities"]["user_mentions"]:
-  if i["screen_name"] != conf["user_name"] and i["screen_name"] != tweet["user"]["screen_name"]:
-   if i[field] not in results:
-    results.append(i[field])
+    if hasattr(tweet, "entities") and tweet.entities.get("user_mentions"):
+        for i in tweet.entities["user_mentions"]:
+            if i["screen_name"] != conf["user_name"] and i["id_str"] != tweet.user:
+                if i.get(field) not in results:
+                    results.append(i.get(field))
     return results
 
-def get_all_users(tweet, conf):
+def get_all_users(tweet, session):
     string = []
- if "retweeted_status" in tweet:
-  string.append(tweet["user"]["screen_name"])
-  tweet = tweet["retweeted_status"]
- if "sender" in tweet:
-  string.append(tweet["sender"]["screen_name"])
+    user = session.get_user(tweet.user)
+    if hasattr(tweet, "retweeted_status"):
+        string.append(user.screen_name)
+        tweet = tweet.retweeted_status
     else:
-  if tweet["user"]["screen_name"] != conf["user_name"]:
-   string.append(tweet["user"]["screen_name"])
-  for i in tweet["entities"]["user_mentions"]:
-   if i["screen_name"] != conf["user_name"] and i["screen_name"] != tweet["user"]["screen_name"]:
+        if user.screen_name != session.db["user_name"]:
+            string.append(user.screen_name)
+        if hasattr(tweet, "entities") and tweet.entities.get("user_mentions"):
+            for i in tweet.entities["user_mentions"]:
+                if i["screen_name"] != session.db["user_name"] and i["screen_name"] != user.screen_name:
                     if i["screen_name"] not in string:
                         string.append(i["screen_name"])
     if len(string) == 0:
-  string.append(tweet["user"]["screen_name"])
+        string.append(user.screen_name)
     return string
 
 def if_user_exists(twitter, user):
     try:
-  data = twitter.show_user(screen_name=user)
+        data = twitter.get_user(screen_name=user)
         return data
- except TwythonError as err:
-  if err.error_code == 404:
+    except TweepError as err:
+        if err.api_code == 50:
             return None
         else:
             return user
 
-def api_call(parent=None, call_name=None, preexec_message="", success="", success_snd="", *args, **kwargs):
- if preexec_message:
-  output.speak(preexec_message, True)
- try:
-  val = getattr(parent.twitter.twitter, call_name)(*args, **kwargs)
-  output.speak(success)
-  parent.parent.sound.play(success_snd)
- except TwythonError as e:
-  output.speak("Error %s: %s" % (e.error_code, e.msg), True)
-  parent.parent.sound.play("error.ogg")
- return val
-
 def is_allowed(tweet, settings, buffer_name):
     clients = settings["twitter"]["ignored_clients"]
- if "sender" in tweet: return True
+    if hasattr(tweet, "sender"): return True
     allowed = True
     tweet_data = {}
- if "retweeted_status" in tweet:
+    if hasattr(tweet, "retweeted_status"):
         tweet_data["retweet"] = True
- if tweet["in_reply_to_status_id_str"] != None:
+    if tweet.in_reply_to_status_id != None:
         tweet_data["reply"] = True
- if "quoted_status" in tweet:
+    if hasattr(tweet, "quoted_status"):
         tweet_data["quote"] = True
- if "retweeted_status" in tweet: tweet = tweet["retweeted_status"]
- source = re.sub(r"(?s)<.*?>", "", tweet["source"])
+    if hasattr(tweet, "retweeted_status"):
+        tweet = tweet.retweeted_status
+    source = tweet.source
     for i in clients:
         if i.lower() == source.lower():
             return False
     return filter_tweet(tweet, tweet_data, settings, buffer_name)
 
 def filter_tweet(tweet, tweet_data, settings, buffer_name):
- if "full_text" in tweet:
+    if hasattr(tweet, "full_text"):
         value = "full_text"
     else:
         value = "text"
@@ -210,24 +195,34 @@ def filter_tweet(tweet, tweet_data, settings, buffer_name):
             if allow_replies == "False" and "reply" in tweet_data:
                 return False
             if word != "" and settings["filters"][i]["if_word_exists"]:
-    if word in tweet[value]:
+                if word in getattr(tweet, value):
                     return False
             elif word != "" and settings["filters"][i]["if_word_exists"] == False:
-    if word not in tweet[value]:
+                if word not in getattr(tweet, value):
                     return False
             if settings["filters"][i]["in_lang"] == "True":
-    if tweet["lang"] not in settings["filters"][i]["languages"]:
+                if getattr(tweet, lang) not in settings["filters"][i]["languages"]:
                     return False
             elif settings["filters"][i]["in_lang"] == "False":
-    if tweet["lang"] in settings["filters"][i]["languages"]:
+                if tweet.lang in settings["filters"][i]["languages"]:
                     return False
     return True
 
 def twitter_error(error):
- if error.error_code == 403:
+    if error.api_code == 179:
         msg = _(u"Sorry, you are not authorised to see this status.")
- elif error.error_code == 404:
+    elif error.api_code == 144:
         msg = _(u"No status found with that ID")
     else:
-  msg = _(u"Error code {0}").format(error.error_code,)
+        msg = _(u"Error code {0}").format(error.api_code,)
     output.speak(msg)
+
+def expand_urls(text, entities):
+    """ Expand all URLS present in text with information found in entities"""
+    if entities.get("urls") == None:
+        return text
+    urls = find_urls_in_text(text)
+    for url in entities["urls"]:
+        if url["url"] in text:
+            text = text.replace(url["url"], url["expanded_url"])
+    return text

src/test/__init__.py

@@ -0,0 +1 @@
+# -*- coding: utf-8 -*-

src/test/test_cache.py

@@ -0,0 +1,200 @@
+# -*- coding: utf-8 -*-
+""" Test case to check some of the scenarios we might face when storing tweets in cache, both loading into memory or rreading from disk. """
+import unittest
+import os
+import paths
+import sqlitedict
+import shutil
+# The base session module requires sound as a dependency, and this needs libVLC to be locatable.
+os.environ['PYTHON_VLC_MODULE_PATH']=os.path.abspath(os.path.join(paths.app_path(), "..", "windows-dependencies", "x86"))
+os.environ['PYTHON_VLC_LIB_PATH']=os.path.abspath(os.path.join(paths.app_path(), "..", "windows-dependencies", "x86", "libvlc.dll"))
+from sessions import base
+
+class cacheTestCase(unittest.TestCase):
+
+    def setUp(self):
+        """ Configures a fake session to check caching objects here. """
+        self.session = base.baseSession("testing")
+        if os.path.exists(os.path.join(paths.config_path(), "testing")) == False:
+            os.mkdir(os.path.join(paths.config_path(), "testing"))
+        self.session.get_configuration()
+
+    def tearDown(self):
+        """ Removes the previously configured session. """
+        session_folder = os.path.join(paths.config_path(), "testing")
+        if os.path.exists(session_folder):
+            shutil.rmtree(session_folder)
+
+    def generate_dataset(self):
+        """ Generates a sample dataset"""
+        dataset = dict(home_timeline=["message" for i in range(10000)], mentions_timeline=["mention" for i in range(20000)])
+        return dataset
+
+    ### Testing database being read from disk.
+
+    def test_cache_in_disk_unlimited_size(self):
+        """ Tests cache database being read from disk, storing the whole datasets. """
+        dataset = self.generate_dataset()
+        self.session.settings["general"]["load_cache_in_memory"] = False
+        self.session.settings["general"]["persist_size"] = -1
+        self.session.load_persistent_data()
+        self.session.db["home_timeline"] = dataset["home_timeline"]
+        self.session.db["mentions_timeline"] = dataset["mentions_timeline"]
+        self.session.save_persistent_data()
+        self.assertIsInstance(self.session.db, sqlitedict.SqliteDict)
+        self.assertTrue(self.session.db.get("home_timeline") != None)
+        self.assertTrue(self.session.db.get("mentions_timeline") != None)
+        self.assertEquals(len(self.session.db.get("home_timeline")), 10000)
+        self.assertEquals(len(self.session.db.get("mentions_timeline")), 20000)
+        self.session.db.close()
+
+    def test_cache_in_disk_limited_dataset(self):
+        """ Tests wether the cache stores only the amount of items we ask it to store. """
+        dataset = self.generate_dataset()
+        self.session.settings["general"]["load_cache_in_memory"] = False
+        self.session.settings["general"]["persist_size"] = 100
+        self.session.load_persistent_data()
+        self.session.db["home_timeline"] = dataset["home_timeline"]
+        self.session.db["mentions_timeline"] = dataset["mentions_timeline"]
+        # We need to save and load the db again because we cannot modify buffers' size while the database is opened.
+        # As TWBlue reads directly from db when reading from disk, an attempt to modify buffers size while Blue is reading the db
+        # Might cause an out of sync error between the GUI lists and the database.
+        # So we perform the changes to buffer size when loading data during app startup if the DB is read from disk.
+        self.session.save_persistent_data()
+        self.session.db = dict()
+        self.session.load_persistent_data()
+        self.assertIsInstance(self.session.db, sqlitedict.SqliteDict)
+        self.assertTrue(self.session.db.get("home_timeline") != None)
+        self.assertTrue(self.session.db.get("mentions_timeline") != None)
+        self.assertEquals(len(self.session.db.get("home_timeline")), 100)
+        self.assertEquals(len(self.session.db.get("mentions_timeline")), 100)
+        self.session.db.close()
+
+    def test_cache_in_disk_limited_dataset_unreversed(self):
+        """Test if the cache is saved properly in unreversed buffers, when newest items are at the end of the list. """
+        dataset = dict(home_timeline=[i for i in range(20)], mentions_timeline=[i for i in range(20)])
+        self.session.settings["general"]["load_cache_in_memory"] = False
+        self.session.settings["general"]["persist_size"] = 10
+        self.session.load_persistent_data()
+        self.session.db["home_timeline"] = dataset["home_timeline"]
+        self.session.db["mentions_timeline"] = dataset["mentions_timeline"]
+        # We need to save and load the db again because we cannot modify buffers' size while the database is opened.
+        # As TWBlue reads directly from db when reading from disk, an attempt to modify buffers size while Blue is reading the db
+        # Might cause an out of sync error between the GUI lists and the database.
+        # So we perform the changes to buffer size when loading data during app startup if the DB is read from disk.
+        self.session.save_persistent_data()
+        self.session.db = dict()
+        self.session.load_persistent_data()
+        self.assertIsInstance(self.session.db, sqlitedict.SqliteDict)
+        self.assertTrue(self.session.db.get("home_timeline") != None)
+        self.assertTrue(self.session.db.get("mentions_timeline") != None)
+        self.assertEquals(self.session.db.get("home_timeline")[0], 10)
+        self.assertEquals(self.session.db.get("mentions_timeline")[0], 10)
+        self.assertEquals(self.session.db.get("home_timeline")[-1], 19)
+        self.assertEquals(self.session.db.get("mentions_timeline")[-1], 19)
+        self.session.db.close()
+
+    def test_cache_in_disk_limited_dataset_reversed(self):
+        """Test if the cache is saved properly in reversed buffers, when newest items are at the start of the list. """
+        dataset = dict(home_timeline=[i for i in range(19, -1, -1)], mentions_timeline=[i for i in range(19, -1, -1)])
+        self.session.settings["general"]["load_cache_in_memory"] = False
+        self.session.settings["general"]["persist_size"] = 10
+        self.session.settings["general"]["reverse_timelines"] = True
+        self.session.load_persistent_data()
+        self.session.db["home_timeline"] = dataset["home_timeline"]
+        self.session.db["mentions_timeline"] = dataset["mentions_timeline"]
+        # We need to save and load the db again because we cannot modify buffers' size while the database is opened.
+        # As TWBlue reads directly from db when reading from disk, an attempt to modify buffers size while Blue is reading the db
+        # Might cause an out of sync error between the GUI lists and the database.
+        # So we perform the changes to buffer size when loading data during app startup if the DB is read from disk.
+        self.session.save_persistent_data()
+        self.session.db = dict()
+        self.session.load_persistent_data()
+        self.assertIsInstance(self.session.db, sqlitedict.SqliteDict)
+        self.assertTrue(self.session.db.get("home_timeline") != None)
+        self.assertTrue(self.session.db.get("mentions_timeline") != None)
+        self.assertEquals(self.session.db.get("home_timeline")[0], 19)
+        self.assertEquals(self.session.db.get("mentions_timeline")[0], 19)
+        self.assertEquals(self.session.db.get("home_timeline")[-1], 10)
+        self.assertEquals(self.session.db.get("mentions_timeline")[-1], 10)
+        self.session.db.close()
+
+    ### Testing database being loaded into memory. Those tests should give the same results than before
+    ### but as we have different code depending whether we load db into memory or read it from disk,
+    ### We need to test this anyways.
+    def test_cache_in_memory_unlimited_size(self):
+        """ Tests cache database being loaded in memory, storing the whole datasets. """
+        dataset = self.generate_dataset()
+        self.session.settings["general"]["load_cache_in_memory"] = True
+        self.session.settings["general"]["persist_size"] = -1
+        self.session.load_persistent_data()
+        self.session.db["home_timeline"] = dataset["home_timeline"]
+        self.session.db["mentions_timeline"] = dataset["mentions_timeline"]
+        self.session.save_persistent_data()
+        self.session.db = dict()
+        self.session.load_persistent_data()
+        self.assertIsInstance(self.session.db, dict)
+        self.assertTrue(self.session.db.get("home_timeline") != None)
+        self.assertTrue(self.session.db.get("mentions_timeline") != None)
+        self.assertEquals(len(self.session.db.get("home_timeline")), 10000)
+        self.assertEquals(len(self.session.db.get("mentions_timeline")), 20000)
+
+    def test_cache_in_memory_limited_dataset(self):
+        """ Tests wether the cache stores only the amount of items we ask it to store, when loaded in memory. """
+        dataset = self.generate_dataset()
+        self.session.settings["general"]["load_cache_in_memory"] = True
+        self.session.settings["general"]["persist_size"] = 100
+        self.session.load_persistent_data()
+        self.session.db["home_timeline"] = dataset["home_timeline"]
+        self.session.db["mentions_timeline"] = dataset["mentions_timeline"]
+        self.session.save_persistent_data()
+        self.session.db = dict()
+        self.session.load_persistent_data()
+        self.assertIsInstance(self.session.db, dict)
+        self.assertTrue(self.session.db.get("home_timeline") != None)
+        self.assertTrue(self.session.db.get("mentions_timeline") != None)
+        self.assertEquals(len(self.session.db.get("home_timeline")), 100)
+        self.assertEquals(len(self.session.db.get("mentions_timeline")), 100)
+
+    def test_cache_in_memory_limited_dataset_unreversed(self):
+        """Test if the cache is saved properly when loaded in memory in unreversed buffers, when newest items are at the end of the list. """
+        dataset = dict(home_timeline=[i for i in range(20)], mentions_timeline=[i for i in range(20)])
+        self.session.settings["general"]["load_cache_in_memory"] = True
+        self.session.settings["general"]["persist_size"] = 10
+        self.session.load_persistent_data()
+        self.assertTrue(len(self.session.db)==1)
+        self.session.db["home_timeline"] = dataset["home_timeline"]
+        self.session.db["mentions_timeline"] = dataset["mentions_timeline"]
+        self.session.save_persistent_data()
+        self.session.db = dict()
+        self.session.load_persistent_data()
+        self.assertIsInstance(self.session.db, dict)
+        self.assertTrue(self.session.db.get("home_timeline") != None)
+        self.assertTrue(self.session.db.get("mentions_timeline") != None)
+        self.assertEquals(self.session.db.get("home_timeline")[0], 10)
+        self.assertEquals(self.session.db.get("mentions_timeline")[0], 10)
+        self.assertEquals(self.session.db.get("home_timeline")[-1], 19)
+        self.assertEquals(self.session.db.get("mentions_timeline")[-1], 19)
+
+    def test_cache_in_memory_limited_dataset_reversed(self):
+        """Test if the cache is saved properly in reversed buffers, when newest items are at the start of the list. This test if for db read into memory. """
+        dataset = dict(home_timeline=[i for i in range(19, -1, -1)], mentions_timeline=[i for i in range(19, -1, -1)])
+        self.session.settings["general"]["load_cache_in_memory"] = True
+        self.session.settings["general"]["persist_size"] = 10
+        self.session.settings["general"]["reverse_timelines"] = True
+        self.session.load_persistent_data()
+        self.session.db["home_timeline"] = dataset["home_timeline"]
+        self.session.db["mentions_timeline"] = dataset["mentions_timeline"]
+        self.session.save_persistent_data()
+        self.session.db = dict()
+        self.session.load_persistent_data()
+        self.assertIsInstance(self.session.db, dict)
+        self.assertTrue(self.session.db.get("home_timeline") != None)
+        self.assertTrue(self.session.db.get("mentions_timeline") != None)
+        self.assertEquals(self.session.db.get("home_timeline")[0], 19)
+        self.assertEquals(self.session.db.get("mentions_timeline")[0], 19)
+        self.assertEquals(self.session.db.get("home_timeline")[-1], 10)
+        self.assertEquals(self.session.db.get("mentions_timeline")[-1], 10)
+
+if __name__ == "__main__":
+    unittest.main()

src/url_shortener/__init__.py

@@ -1,3 +1,3 @@
 from __future__ import unicode_literals
 from . import shorteners
-from __main__ import *
+from . __main__ import *

src/url_shortener/__main__.py

@@ -1,6 +1,6 @@
 from __future__ import unicode_literals
 from functools import wraps
-import shorteners
+from . import shorteners
 
 
 def service_selecter (func):

src/wxUI/dialogs/attach.py

@@ -41,7 +41,7 @@ class attachDialog(widgetUtils.BaseDialog):
         return (openFileDialog.GetPath(), dsc)
 
     def ask_description(self):
-		dlg = wx.TextEntryDialog(self, _(u"please provide a description"), _(u"Description"), defaultValue="")
+        dlg = wx.TextEntryDialog(self, _(u"please provide a description"), _(u"Description"))
         dlg.ShowModal()
         result = dlg.GetValue()
         dlg.Destroy()

src/wxUI/dialogs/configuration.py

@@ -1,7 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import absolute_import
-from __future__ import unicode_literals
-from builtins import range
 import logging as original_logger
 import wx
 import application
@@ -79,7 +76,7 @@ class proxy(wx.Panel, baseDialog.BaseWXDialog):
         serverBox.Add(self.server, 0, wx.ALL, 5)
         sizer.Add(serverBox, 0, wx.ALL, 5)
         lbl = wx.StaticText(self, wx.ID_ANY, _(u"Port: "))
-  self.port = wx.TextCtrl(self, wx.ID_ANY)
+        self.port = wx.SpinCtrl(self, wx.ID_ANY, min=1, max=65535)
         portBox = wx.BoxSizer(wx.HORIZONTAL)
         portBox.Add(lbl, 0, wx.ALL, 5)
         portBox.Add(self.port, 0, wx.ALL, 5)
@@ -127,6 +124,7 @@ class generalAccount(wx.Panel, baseDialog.BaseWXDialog):
         self.persist_size = wx.TextCtrl(self, -1)
         sizer.Add(PersistSizeLabel, 0, wx.ALL, 5)
         sizer.Add(self.persist_size, 0, wx.ALL, 5)
+        self.load_cache_in_memory = wx.CheckBox(self, wx.NewId(), _("Load cache for tweets in memory (much faster in big datasets but requires more RAM)"))
         self.SetSizer(sizer)
 
 class reporting(wx.Panel, baseDialog.BaseWXDialog):

src/wxUI/dialogs/lists.py

@@ -94,10 +94,10 @@ class editListDialog(createListDialog):
 
     def __init__(self, list, *args, **kwargs):
         super(editListDialog, self).__init__(*args, **kwargs)
-  self.SetTitle(_(u"Editing the list %s") % (list["name"]))
-  self.name.ChangeValue(list["name"])
-  self.description.ChangeValue(list["description"])
-  if list["mode"] == "public":
+        self.SetTitle(_(u"Editing the list %s") % (list.name))
+        self.name.ChangeValue(list.name)
+        self.description.ChangeValue(list.description)
+        if list.mode == "public":
             self.public.SetValue(True)
         else:
             self.private.SetValue(True)

src/wxUI/dialogs/message.py

@@ -221,7 +221,7 @@ class dm(textLimited):
 
     def __init__(self, title, message,  users, *args, **kwargs):
         super(dm, self).__init__()
-  self.createControls(message, title, users)
+        self.createControls(title, message, users)
 #  self.onTimer(wx.EVT_CHAR_HOOK)
 #  self.SetClientSize(self.mainBox.CalcMin())
 

src/wxUI/dialogs/search.py

@@ -1,6 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import absolute_import
-from __future__ import unicode_literals
 import widgetUtils
 from . import baseDialog
 import wx
@@ -14,6 +12,7 @@ class searchDialog(baseDialog.BaseWXDialog):
         self.SetTitle(_(u"Search on Twitter"))
         label = wx.StaticText(panel, -1, _(u"&Search"))
         self.term = wx.TextCtrl(panel, -1, value)
+        self.term.SetFocus()
         dc = wx.WindowDC(self.term)
         dc.SetFont(self.term.GetFont())
         self.term.SetSize(dc.GetTextExtent("0"*40))

src/wxUI/dialogs/trends.py

@@ -1,7 +1,4 @@
 # -*- coding: utf-8 -*-
-from __future__ import absolute_import
-from __future__ import unicode_literals
-# -*- coding: utf-8 -*-
 from . import baseDialog
 import wx
 
@@ -11,8 +8,7 @@ class trendingTopicsDialog(baseDialog.BaseWXDialog):
         panel = wx.Panel(self)
         sizer = wx.BoxSizer(wx.VERTICAL)
         self.SetTitle(_(u"View trending topics"))
-  label = wx.StaticText(panel, -1, _(u"Trending topics by"))
-  sizer.Add(label, 0, wx.ALL, 5)
+        label = wx.StaticText(panel, wx.NewId(), _(u"Trending topics by"))
         self.country = wx.RadioButton(panel, -1, _(u"Country"), style=wx.RB_GROUP)
         self.city = wx.RadioButton(panel, -1, _(u"City"))
         radioSizer = wx.BoxSizer(wx.HORIZONTAL)

updates/snapshots.json

@@ -1,5 +1,5 @@
-{"current_version": "1",
+{"current_version": "6",
 "description": "Snapshot version.",
 "date": "unknown",
 "downloads":
-{"Windows32": "https://twblue.es/snapshot.zip"}}
+{"Windows32": "https://twblue.es/pubs/snapshot.zip"}}