Blog

  • Announcing MySQL Multi-Source Replication Utility

    By Nuno Mariz, on 1 April 2014 @ 15:05

    The MySQL Utilities team is pleased to announce a new MySQL utility, mysqlrplms, that allows users to setup multi-source replication. This utility is available for server versions 5.6.9 and later with GTIDs enabled. The new utility is included in MySQL Utilities release-1.4.2 RC.

    What is multi-source replication?

    Multi-source replication is often regarded as a means to aggregate and consolidate different streams of data (from master servers - also called sources) into one single server (slave) instance. This is useful for consolidating non-conflicting data changes into a central server. Not only does this allow users to perform backups of the consolidated data, it also permits users to execute queries that may span the datasets.

    Requirements and assumptions

    • The mysqlrplms utility supports MySQL 5.6.9 and higher version servers with GTIDs enabled.
    • There must not be any overlap or data conflicts for the data from different masters. For example, there are no updates to the same object from multiple masters.
    • Replication must be asynchronous.
    • A round-robin scheduling is used to setup replication among all masters and slave.

    Note: a POSIX system is required for using the daemon option.

    Examples

    Basic multi-source replication setup:

    shell> mysqlrplms --slave=root:pass@host1:3306 \
                      --masters=root:pass@host2:3306,root:pass@host3:3306
    # Starting multi-source replication...
    # Press CTRL+C to quit.
    # Switching to master 'host2:3306'.
    # master on localhost: ... connected.
    # slave on localhost: ... connected.
    #
    # Current Master Information:
    +-------------------+-----------+---------------+-------------------+
    | Binary Log File   | Position  | Binlog_Do_DB  | Binlog_Ignore_DB  |
    +-------------------+-----------+---------------+-------------------+
    | clone-bin.000001  | 594       | N/A           | N/A               |
    +-------------------+-----------+---------------+-------------------+
    # GTID Executed Set: 0cd1200c-b5a2-11e3-a342-28d244017f26:1-2
    #
    # Health Status:
    +------------+-------+---------+--------+------------+---------+
    | host       | port  | role    | state  | gtid_mode  | health  |
    +------------+-------+---------+--------+------------+---------+
    | host2      | 3306  | MASTER  | UP     | ON         | OK      |
    | host1      | 3306  | SLAVE   | UP     | ON         | OK      |
    | host3      | 3306  | MASTER  | UP     | ON         | OK      |
    +------------+-------+---------+--------+------------+---------+
    (...)
    # Switching to master host3:3306'.
    # master on localhost: ... connected.
    # slave on localhost: ... connected.
    #
    # Current Master Information:
    +-------------------+-----------+---------------+-------------------+
    | Binary Log File   | Position  | Binlog_Do_DB  | Binlog_Ignore_DB  |
    +-------------------+-----------+---------------+-------------------+
    | clone-bin.000001  | 594       | N/A           | N/A               |
    +-------------------+-----------+---------------+-------------------+
    # GTID Executed Set: c30a1796-b998-11e3-bd1a-28d244017f26:1-2
    #
    # Health Status:
    +------------+-------+---------+--------+------------+---------+
    | host       | port  | role    | state  | gtid_mode  | health  |
    +------------+-------+---------+--------+------------+---------+
    | host3      | 3306  | MASTER  | UP     | ON         | OK      |
    | host1      | 3306  | SLAVE   | UP     | ON         | OK      |
    | host2      | 3306  | MASTER  | UP     | ON         | OK      |
    +------------+-------+---------+--------+------------+---------+
    (...)
    

    Multi-source replication setup using different report values:

    shell> mysqlrplms --slave=root:pass@host1:3306 \
                      --masters=root:pass@host2:3306,root:pass@host3:3306 \
                      --report-values=health,gtid,uuid
    

    Start multi-source replication running as a POSIX daemon:

    shell> mysqlrplms --slave=root:pass@host1:3306 \
                      --masters=root:pass@host2:3306,root:pass@host3:3306 \
                      --log=rplms_daemon.log --pidfile=rplms_daemon.pid \
                      --daemon=start
    

    Restart a multi-source replication running as a POSIX daemon:

    shell> mysqlrplms --slave=root:pass@host1:3306 \
                      --masters=root:pass@host2:3306,root:pass@host3:3306 \
                      --log=rplms_daemon.log --pidfile=rplms_daemon.pid \
                      --daemon=restart
    

    Stop a multi-source replication running as a POSIX daemon:

    shell> mysqlrplms --slave=root:pass@host1:3306 \
                      --masters=root:pass@host2:3306,root:pass@host3:3306 \
                      --log=rplms_daemon.log --pidfile=rplms_daemon.pid \
                      --daemon=stop
    

    Send us your feedback

    MySQL Utilities release-1.4.2 RC is available for download from the following links:

    The documentation of MySQL Utilities can be obtained from the following link: http://dev.mysql.com/doc/index-gui.html

    Contributing Ideas:

  • New year, new job

    By Nuno Mariz, on 14 December 2012 @ 10:02

    Starting from January 2013, I'll be working at Oracle on the MySQL Utilities Team as a Software Developer.
    It's a new chapter on my professional career and a unique opportunity for doing what I like the most, which is Python programming on an open source project.

    I'm leaving InfoPortugal S.A., a special company that was my home for the last 6 years. It was a pleasure working with such a talented team. A special Thanks to Alexandre Gomes, a great professional with an incomparable commitment to the company.

  • PostgreSQL and PostGIS installation from source on Mac OS X Lion

    By Nuno Mariz, on 19 June 2012 @ 11:59

    This is a cookbook for installing PostgreSQL and PostGIS on Mac OS X Lion from source, as alternative you can install the binaries from EnterpriseDB or KyngChaos.

    You don't have to install all packages but if you're using Django will be useful.

    Requirements

    Xcode with "Command Line Tools": Installation via Xcode > Preferences > Downloads.

    PostgreSQL

    1. Create the postgres user via System Preferences(for simplicity), use "postgres" as username.
    2. Compiling and installing PostgreSQL:
      $ curl -O http://ftp.postgresql.org/pub/source/v9.1.3/postgresql-9.1.3.tar.gz
      $ tar xzfp postgresql-9.1.3.tar.gz
      $ cd postgresql-9.1.3
      $ ./configure CC=/usr/bin/clang CXX=/usr/bin/clang++ "CFLAGS=-arch x86_64" "LDFLAGS=-arch x86_64" "CXXFLAGS=-arch x86_64"
      $ make
      $ sudo make install
      $ cd ..
      
    3. Create PostgreSQL Database Cluster:
      $ sudo mkdir /usr/local/pgsql/data
      $ sudo chown postgres /usr/local/pgsql/data
      $ sudo -u postgres /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
      
    4. Start PostgreSQL Server:
      $ sudo -u postgres /usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data start
      
    5. After installation add to .profile or .zshrc:
      export PATH=/usr/local/pgsql/bin:$PATH
      

    PROJ4

    PROJ.4 is a library for converting geospatial data to different coordinate reference systems.

    $ curl -O http://download.osgeo.org/proj/proj-4.8.0.tar.gz
    $ curl -O http://download.osgeo.org/proj/proj-datumgrid-1.5.zip
    $ tar xzfp proj-4.8.0.tar.gz
    $ cd proj-4.8.0/nad
    $ unzip ../../proj-datumgrid-1.5.zip
    $ cd ..
    $ ./configure CC=/usr/bin/clang CXX=/usr/bin/clang++ "CFLAGS=-arch x86_64" "LDFLAGS=-arch x86_64" "CXXFLAGS=-arch x86_64"
    $ make
    $ sudo make install
    $ cd ..
    

    GEOS

    GEOS is a C++ library for performing geometric operations.

    $ curl -O http://download.osgeo.org/geos/geos-3.3.3.tar.bz2
    $ tar xjfp geos-3.3.3.tar.bz2
    $ cd geos-3.3.3
    $ ./configure CC=/usr/bin/clang CXX=/usr/bin/clang++ "CFLAGS=-arch x86_64" "LDFLAGS=-arch x86_64" "CXXFLAGS=-arch x86_64"
    $ make
    $ sudo make install
    $ cd ..
    

    GDAL

    GDAL is an open source geospatial library that has support for reading most vector and raster spatial data formats. GEOS and PROJ.4 should be installed prior to building GDAL.

    $ curl -O http://download.osgeo.org/gdal/gdal-1.9.1.tar.gz
    $ tar xzfp gdal-1.9.1.tar.gz
    $ cd gdal-1.9.1
    $ ./configure CC=/usr/bin/clang CXX=/usr/bin/clang++ "CFLAGS=-arch x86_64" "LDFLAGS=-arch x86_64" "CXXFLAGS=-arch x86_64"
    $ make
    $ sudo make install
    $ cd ..
    

    PostGIS

    Spatial extensions for PostgreSQL.

    $ curl -O http://postgis.refractions.net/download/postgis-1.5.4.tar.gz
    $ tar xzfp postgis-1.5.4.tar.gz
    $ cd postgis-1.5.4
    $ ./configure CC=/usr/bin/clang CXX=/usr/bin/clang++ "CFLAGS=-arch x86_64" "LDFLAGS=-arch x86_64" "CXXFLAGS=-arch x86_64"
    $ make
    $ sudo make install
    
  • Gabriela Barbosa Mariz

    By Nuno Mariz, on 12 March 2012 @ 16:36
    Gabriela
    Gabriela Barbosa Mariz was born at 16h36, March 12, 2012. She weighed 3.710 Kg and was 51 cm long.
    Gabriela and Marta are doing well.
  • ITJobs Android App

    By Nuno Mariz, on 27 February 2012 @ 13:38

    Just released the ITJobs Android app, it was a fun(sometimes) project to make.
    The application supports devices with Android 1.6 or greater. For those people that says Android fragmentation is a non issue, I've to say they are wrong, it turns the life of developers extremely difficult and boring.
    Too many versions and resolutions to take care about. I had the feeling of continuous patching just to support the largest devices that I could.
    I've also found some nonsense and sometimes ridiculous bugs on Android SDK, I'll post about this later.

    But overall it's not a bad platform to develop. What I've enjoyed the most was the flexibility about the XML layouts and shapes, the Eclipse IDE does the job and I think Java is a nice programming language.
    Please send me some feedback, the app is available on Android Market:

    ITJobs Android Market


    ITJobs Android Screenshot
  • Happy new year

    By Nuno Mariz, on 31 December 2011 @ 23:03

    No resolutions, just goals and dreams. Have a healthy baby girl, love, health, run half marathon in 2h and success for my upcoming projects.


    Happy new year!

  • Announcing Daddy 2.0

    By Nuno Mariz, on 18 August 2011 @ 14:46
    Daddy 2.0

    The project is under development and my co-worker is doing her best to provide the most beautiful layout.

    For my english speaking readers the photo says: "Pregnant 3+"

  • Believe in your ideas!

    By Nuno Mariz, on 3 May 2011 @ 17:16
    Later in 2007 I had almost 200 feeds subscriptions on my Google Reader account and I had a problem, sometimes I didn't have the time to read a full article and I wanted to read it later.
    At that time I was not aware of a tool that would help me with this problem and my solution was to add a bookmark to Delicious with a readlater tag.
    It worked, but I didn't want to mess my bookmarks with links to articles that I wanted to read later.
    I felt that I needed a tool for the job and for about a week in my spare time I built a simple website to manage my online readings. It was simple, It had a bookmarklet that I could call to save a link of the page that I was visiting, add a title, description, tags and the ability to save it privately.
    On February, 8th, 2008 I announced on my blog the release of Fullread. The feedback was a mix of "I use delicious with a tag for that" and "Looks nice, but not very useful".
    At that time I built the tool mainly for myself and I truly believed that there was a need for this kind of thing, but the majority of the feedback was not that great. I continued using the tool but I didn't add more features to it. If the people didn't liked why bother?
    Well, 25 days later Marco Arment released Instapaper and posted on his blog. Marco went further and added an iPhone app, that was in my opinion the killer feature.
    Right now Instapaper is the market leader in this category, it's a fantastic tool with a continuous development and I use it every day.

    So, why Fullread had no success compared to Instapaper?
    In my opinion the factors were:
    • Mainly because Marco made a great job.
    • Marco was the CTO of Tumblr and was well known by the community, this helped with the media coverage. I'm a simple software engineer in Portugal.
    • I abandoned Fullread development.

    What's the future of Fullread?
    The answer is not simple. It’s very difficult to compete with Instapaper or Read it Later at this time of the game. But, in the name of pride, I'll eventually improve/rebuild Fullread, maybe with a different approach.

    So, is there a lesson to take from all this, I ask?
    Yes: Believe in yourself, believe in your ideas!
  • My daily tools

    By Nuno Mariz, on 23 March 2011 @ 13:00

    Hardware

    At home

    At work

    • Computer 1: iMac 21.5" - Intel Core 2 3.0Ghz, 4Gb RAM and 500Gb HD.
    • Computer 2: Ubuntu Linux - Intel Quad-Core 2.4Ghz, 4Gb RAM, WD Raptor 150Gb + Samsung HD502 750Gb. Used for testing and processing maps.
    • Phone: HTC HD2 with a custom Android 2.3 Gingerbread ROM.

    Software

    For development

    • Emacs: I'm using it for all web development, scripting and text editing.
    • Xcode: For iOS development.
    • Eclipse: For Android development.
    • Firebug: For web development.
    • Mercurial: The distributed source control management of my choice.
    • Virtualenv: Virtual Python environment builder.
    • PostgreSQL: The relational database of my choice.
    • PostGIS: Adds support for geographic objects to the PostgreSQL relational database.
    • MongoDB: A document-oriented database.

    Other tools

    • Evernote: My information organization tool for all my projects ideas, notes etc.
    • Dropbox: For file sync, no more pen drives!
    • Reeder: I use both iPhone and desktop versions to consume RSS feeds.
    • Twitterrific: I use both iPhone and desktop versions, it's the Twitter client that's fast enough on my iPhone 3G.

    What's yours?

  • New website

    By Nuno Mariz, on 16 March 2011 @ 19:32
    The new website is up. After almost 2 years, I'm blogging again.
    The reason for this gap It's a mix of Twitter usage and laziness.
    In the migration process I've messed up the RSS feed, solved, my apologies that.